Ruling the Swarm (of Tests) with Codeception

What happens if you have more tests then zergs in the swarm? Really much more. You should find a way to control and manage them. In this article I’ll share some hints and best practices you may use writing acceptance tests. I’ll use Codeception testing framework to illustrate the best practices, but surely, they can be ported to any acceptance framework you use for testing.

Into Codeception

Designing and building a great web application means that our workflow and architecture allow for constant improvement to the product. In order to make our products stable and to have confidence that new commits have no adverse-effect on existing code, we use automated testing practices. And just as we need a proper architecture for our application, we need to design a proper architecture for our testing platform.

A testing platform is a complete infrastructure over our tests. And when I say tests, I don’t mean only unit tests. Think wider. The application also needs to be tested in a browser with Selenium (or an alternative). So we need unit tests, Selenium tests, probably API tests, and all of them should be executed together. Somehow. I’d like to introduce you Codeception – a modern PHP testing framework built on top of Symfony2 components, Mink, and PHPUnit.

Codeception includes more than 20 modules to test over popular PHP frameworks, APIs (REST, SOAP), data stores (MySQL, PostgreSQL, MongoDB, etc) and web with Selenium WebDriver. Check it out on the official web site: codeception.com

And what better way to run all of these tests than on an Continuous Integration server after each commit? I recommend using Codeception as the main block for your test automation platform; Codeception can execute Selenium tests, API tests, and PHPUnit tests all in one and generate complete HTML and XML reports for CI systems. It can even generate a complete code coverage report for all of the tests.

It’s pretty easy to start writing tests with Codeception. You can teach your QAs to write such tests since only a very basic knowledge of PHP is required to start. I’ll leave you to discover the basics of Codeception on your own, since what I want to tell you here is how Codeception can help you build a better test automation platform.

Hive of Tests

It’s not enough to say “We are writing tests” or “We practice TDD/BDD”. What you should be aiming for is the creation of a solid test automation platform. It doesn’t really matter if you’re writing your tests before code, or code before tests, or tests described as a story (in BDD-way), or tests as code. What really matters is how you manage all of these tests.

There may be more then 1,000 or 10,000 or 100,000 tests, and they may run for a three or four hours. You need keep the code for your tests clean and simple. Tests should be easy to refactor (as they constantly change with application) and they should be easy to extend. Basically, you should follow the same best practices you use for writing production code when writing testing code.

A Basic Test

Here is a sample acceptance test in Codeception. In this scenario I’m sending an invite to a friend from a website. To test this I need to be logged in, generate an invitation code, and then try to register a new user with the code. Here is how the scenario could look in Codeception:

<?php
$I = new WebGuy($scenario);
$I->wantTo('generate invitation code and sign up using it');
$I->amOnPage('/login');
$I->fillField('Username','tester1');
$I->fillField('Password','123456');
$I->click('Sign In');
$I->see('Hello, tester1');
$I->click('Invite');
$I->see('Copy this invitation code');
$code = $I->grabTextFrom('#invitation_code');
$I->click('Logout');
$I->amOnPage('/register');
$I->see('You can sign up if you have invitation code!', 'h1');
$I->fillField('Invitation Code', $code);
$I->fillField('Username','tester2');
$I->fillField('Password','654321');
$I->click('#invite_form input[type=submit]');
$I->see('Hello, tester2');

Even if you’re not familiar with Codeception, this test should be pretty readable; all actions are described from a user’s perspective and no special technical terms are used here. This test may look simple and clean, but if you have such tests then you should consider refactoring it. And here is why…

Design Pattern: Page Object

You should avoid hard-coded values directly in your tests whenever you can as this will make the tests more stable to changes. Suppose similar login and signup forms are used in other tests. If the username field was renamed to “Login” then you’d need to rewrite the tests that use login. We can use the Page Object pattern and CSS/XPath to locate elements on a page. The same goes for the Invite page.

A Page Object is an entity that represents a page (or several typical pages) of your site, a can be written as a class with static properties and methods.

<?php
class InvitePage
{
    // URL of a page
    public static $URL = '/register';
    // This properties define a UI map for Login Page
    public static $codeField = "Invitation Code";
    public static $usernameField = "Username";
    public static $passwordField = "Password";
    public static $submitButton = "#invite_form input[type=submit]";
}

Now part of the test can be rewritten in this manner:

<?php
$I->amOnPage(InvitePage::$URL);
$I->fillField(InvitePage::$codeField, $code);
$I->fillField(InvitePage::$usernameField,'tester2');
$I->fillField(InvitePage::$passwordField,'654321');
$I->click(InvitePage::$submitButton);

In this case, all tests visiting the Invite page use the same variables for locating elements on the page. If any element changes its position, you can update the InvitePage class as needed without touching the tests themselves.

Design Pattern: Controller

We’ve removed some hardcoded variables from the test, but what happens if you introduce two-factor authentication to the web application you’re developing. What does it mean for your tests? It means that you’ll need to update all of the tests that requires an authorized user to run. I will remind you: there may be thousands and of tests. It would be natural to move some scenario logic into a class so we can reuse the log in/out and signup scenarios in other tests.

I asked some automation engineer friends how they refer to such a class. They used the term Step Object (which is common in BDD), but I’d rather call it a Controller here because of the similarity it has with the controllers we use in web applications. There are no strict rules stating what page objects or step objects should look like; you are free to build them as you wish. Here is an example controller I would use for Codeception test:

<?php
class UserController
{
    protected $user;

    public function __construct(WebGuy $I) {
        $this->user = $I;
    }

    public function login($username, $password) {
        $this->user->amOnPage(LoginPage::$URL);
        $this->user->fillField(LoginPage::$usernameField, $username);
        $this->user->fillField(LoginPage::$passwordField, $password);
        $this->user->click(LoginPage::$submitButton);
        $this->user->see("Hello, $username");
    }

    public function logout() {
        $this->user->click("Logout");
    }

    public function registerWithInviteCode($code, $username, $password) {
        $this->user->amOnPage(InvitePage::$URL);
        $this->user->fillField(InvitePage::$codeField, $code);
        $this->user->fillField(InvitePage::$usernameField, $username);
        $this->user->fillField(InvitePage::$passwordField, $password);
        $this->user->click(InvitePage::$submitButton);
    }
}

Now the code has logical blocks and the scenario will look much cleaner:

<?php
$I = new WebGuy($scenario);
$U = new UserController($I);
$I->wantTo('generate invitation code and sign up using it');
$U->login('tester1', '123456');
$I->click('Invite');
$I->see('Copy this invitation code');
$code = $I->grabTextFrom('#invitation_code');
$U->logout();
$U->registerWithInviteCode($code, 'tester2', '654321');
$I->see('Hello, tester2');

By introducing controllers and page objects, we’ve gotten rid of most hardcoded values and common scenario steps. It can log users in from other tests, and that’s really useful as probably most of your application’s functionality is available only to authorized users. You can write 10 or 100 or even 1,000 tests, but using page objects and controllers they won’t make a mess and will be easy to support.

Report to the Commander

Because of the refactoring, the test code has probably lost some of its readability to non-developers. Luckily, Codeception can help with that, too. Codeception will show you all scenario steps that were executed when running the test:

$ ./codecept.phar run acceptance --steps
Codeception PHP Testing Framework v1.5.5
Powered by PHPUnit 3.7.14 by Sebastian Bergmann.

Suite acceptance started
Trying to generate invitation code and sign up using it (InviteCept.php)
Scenario:
* I am on page '/login'
* I fill field 'Username','tester1'
* I fill field 'Password','123456'
* I click 'Sign In'
* I see 'Hello, tester1'
* I click 'Invite'
* I see 'Copy this invitation code'
* I grab text from '#invitation_code'
* I click 'Logout'
* I am on page '/register'
* I see 'You can sign up if you have invitation code!', 'h1'
* I fill field 'Invitation Code', 'iuSia89ak'
* I fill field 'Username','tester2'
* I fill field 'Password','654321'
* I click '#invite_form input[type=submit]'
* I see 'Hello, tester2'

Time: 10 seconds, Memory: 26.00Mb
OK (1 test, 4 assertions)

In this case the output is readable. This scenario can even be shown to a non-technical person, to a manager, or a business analyst, and they’ll understand what is happening. Codeception allows you to generate HTML reports written in a similar manner. They can be shown to managers or business analysts to demonstrate what and how you you’re your application.

Conclusion

Nowadays test automation engineers prefer Java and Ruby, but as a PHP developer you’re undoubtedly interested in testing platforms that play well with your PHP code and is written in PHP itself. Codeception is a great tool to easily start acceptance testing, and it’s flexible enough to build a solid test automation platform.

But no matter what framework you use, keep in mind that it’s not enough just to write tests. To make tests clean, stable, and maintainable, use the KISS and DRY principles you already know. Design patterns like Page Object and Controller (Step Object) will help you in that.

Image via Fotolia