Make Sure Your Functions Function: Unit Testing

Hopefully you’ve checked out our new contributor guide and you’re ready to submit your first patch — thanks! But before you send your .diff out into the world, run a unit test to make sure everything’s working as it should.

Never written or run a unit test? Here’s a quick get-started and best practices guide.

Why Unit Test?

Moving forward (beyond Jetpack 2.3.4) all new functionality must be accompanied by a successful unit test. Unit tests are small snippets of code that ensure a the new code is working as expected.

Unit testing is a crucial piece of your contribution to Jetpack: it verifies both that Jetpack works as expected, and that the updates won’t break the existing code. As we work together to build new features and zap the bugs, unit tests provide ongoing quality assurance that keep Jetpack strong and reliable.

Set Up a Unit Testing Environment

To run local tests, you’ll need to install PHPUnit and make sure you’ve got a few other prerequisites set up — here’s an overview. (You can get the detailed version here.)

Install PHPUnit

PHPUnit is distributed via pear. To ensure you are able to retrieve all the necessary components you will want to add the following pear channels:

sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear channel-discover pear.symfony-project.com

When you’ve got the files downloaded, run an install command to get everything set up:

sudo pear install phpunit/PHPUnit


Get the WordPress testing codebase

Unit testing for plugins uses the same codebase as unit testing WordPress core itself. Pull a copy with:

svn checkout http://unit-tests.svn.wordpress.org/trunk wordpress-tests

Get Jetpack

Download Jetpack’s code, and make sure it’s saved in wordpress/wp-content/plugins.

You have two options for getting the code, depending on what version control software you prefer:

Subversion: download the code from the WordPress.org plugin repository using trunk:

svn checkout http://plugins.svn.wordpress.org/jetpack/trunk jetpack

GitHub: Clone the GitHub fork maintained by @blobaugh. (This fork also runs automated tests with Travis-CI.)

git clone git@github.com:blobaugh/Jetpack.git
or
git clone https://github.com/blobaugh/Jetpack.git

Writing tests

Once you’ve got a testing environment and the codebase, time to write a test! Tests live in trunk/tests on WordPress.org, or in the tests folder on GitHub.

Create a new test file

When creating new test files, all files should be named ‘test_.php’ (where “test” is the same as the file containing the code being tested). Tests live inside classes named:

class WP_Test_ extends WP_UnitTestCase

where “Test” is the same as the class containing the code to be tested

Write a good test

Here are some good general guidelines for writing unit tests:

• Tests should be specific — test one feature at a time.
• Tests should be orthogonal (independent) of all other tests. They shouldn’t rely on other tests to run or pass.
• Tests should be designed to pass — you’re trying to prove success, not failure.
• Tests should be repeatable.
• One function may (and probably should) have multiple tests associated with it; one test for each possible condition.
• Create a new test every time you find a bug to make sure it’ll be caught if it shows up again in the future.
• Use static data in tests to get clear results. Dynamic data can introduce hard-to-diagnose failures
• When testing external services, it’s better to re-create the service details (data received, API state, etc) than to connect to the actual service.
• Use descriptive names for tests so what’s being tested is clear to everyone; long method names aren’t a problem.

Running tests

Now that you have unit testing setup and you’ve written some specific, clear new tests, it’s time to run ’em! You should always run the tests locally to ensure your code is working properly before creating a patch or pushing the code into trunk.

Testing locally

Make sure you are in the Jetpack plugin directory, and type:

phpunit

Testing with Travis-CI

Testing with Travis-CI is as easy as committing the code back to @blobaugh’s GitHub fork and visiting https://travis-ci.org/blobaugh/Jetpack. A new test should kick off shortly.

Failing tests

So your test failed. Now what?

Most importantly, no cheating! Patches submitted with failing tests will not be accepted into trunk. Please don’t try to get around this by removing the test — an unstable patch hurts every site using Jetpack, so we need to be strict about this.

Luckily, PHP Unit tells you which test failed, so you can use that data and update your code to solve the issue. If the failure is with a previously existing function, chances are that your function is interacting with it in a way that is causes failure.

If you’ve tried everything you can think of and your tests continue to fail, ping us directly — we’re happy to help troubleshoot.

Want to be a unit testing master? Check out these fine resources:

• Ben Lobaugh’s Jetpack fork
  • Useful for automated testing with Travis-CI
• Automated tests with Travis-CI – based on Ben Lobaugh’s fork
  • Visualize and share pass/fail reports from Ben’s fork
• Nikolay Bachiyski introduces Unit Testing at WordCamp
  • Nikolay introduces the unit testing process with a live demo
• Nikolay Bachiyski’s WordCamp slides
• XUnit Patterns book
  • THE book to read on creating good tests and testing patterns
• PHP Unit testing documentation manual
  • The Codex of building tests with PHP Unit

---