In this post, we'll cover the basics of unit tests and walk through the process determining what types of tests you should write for your code. We'll also write a few tests to show just how simple most unit tests are.
The aim here is to make tests a bit more familiar and less daunting and mysterious. Unit tests are good. I'm by no means an expert on software testing. I'm just someone that's discovered the joy and freedom that writing tests provides.
If you are an expert on software testing and notice I'm wrong, please tell me!
Onward!
What are unit tests?
If you’re not familiar with unit tests, they’re essentially software written to test discrete units of functionality in a system.
Unit tests test the expected behavior of your software as it exists (or existed) at a specific point in time. They make sure that everything works as expected based on the rules established for the software at the time it was written.
If those rules change -- and your software along with them -- your tests will fail. When they fail, you'll gain insight into how the change affects the rest of the system.
For example, if a change is made in code for your site’s user profile management, you'll want to make sure that the change does not affect the behavior of your site’s payment processing code.
Tests can save time and money by acting as the first line of defense in catching problems before they result in a late night emergency or catastrophic failure. When things go wrong, they can also serve as the first response team. You can run your unit tests to see if any rules/expectations have been broken by some recent change, either in your code or some external API dependency, and fix the bug introduced by the change.
Sometimes the tests become outdated. Again, the tests will fail, letting you know that you’re no longer testing for the right thing. When this happens, you should update your tests to make sure they are in line with the new rules for your software.
Getting started
Unit tests are rarely difficult to write or run if you’re using a testing framework (e.g., phpunit).
You can write your tests before you write your business code if Test-driven Development (TDD) is your thing or you can write them after.
It doesn’t matter much when you write them as long as you have them.
The brilliant thing is that almost all tests you write will be stupid simple.
Tests usually assert some condition is true (or false, depending on what outcome you expect).
On top of that, most testing frameworks have convenience functions built in for performing these types of tests and several others, including testing the expected output of a function (i.e., what the function prints to the screen).
The three test functions we've used most frequently when writing tests for Largo are assertEquals
, assertTrue
and expectOutputString
.
Our stupid simple example plugin
So, let’s say that we want to write tests for a WordPress plugin. Our plugin simply prints the message “Hello world!” in the top corner of the WordPress dashboard (sound familiar?).
This is the extent of the code:
<?php
/**
* @package Hello_World
* @version 0.1
*
* Plugin Name: Hello World!
* Plugin URI: https://gist.github.com/rnagle/d561bd58504a644e9657
* Description: Just a simple WordPress plugin that prints "Hello world!" in the top corner of the WordPress dashboard.
* Author: Ryan Nagle
* Version: 0.1
**/
function hello_world() {
return "Hello world!";
}
function hello_world_markup() {
echo "<p id='hello-world'>" . hello_world() . "</p>";
}
add_action('admin_notices', 'hello_world_markup');
function hello_world_css() {
$x = is_rtl() ? 'left' : 'right';
echo "
<style type='text/css'>
#hello-world {
float: $x;
padding-$x: 15px;
padding-top: 5px;
margin: 0;
font-size: 11px;
}
</style>
";
}
add_action('admin_head', 'hello_world_css');
Couple of things to note:
- We have one function --
hello_world
-- that returns a string. - Two other functions --
hello_world_markup
andhello_world_css
-- that echo strings but have no return statements. This means we'll have to check the output of these functions rather than the return value to properly test them. - Note that
hello_world_markup
relies onhello_world
to provide the the “Hello world!” message.
So, how do we test this plugin? Well, if you don’t have your test framework installed and configured, you’ll want to get that ready and raring. The details are beyond the scope of what we'll cover here, but for an overview of setting up unit tests for WordPress plugins and themes, see Will Haynes’ Unit Testing Themes and Plugins in WordPress.
The INN Nerds also have a collection of deploy tools to make developing WordPress sites and writing tests much easier. You can read about using the deploy tools for testing themes and plugins here: Updates to INN’s Deploy Tools.
Disclaimer: getting unit tests working WordPress can be daunting at first. It’s a lot to digest and may take some time to set up if you’re coming to this cold, so be patient. It’ll be worth it.
If you’re really dedicated to the cause and need a hand, reach out to us.
How do I write a unit test?
With that out of the way, let’s design some unit tests for our “Hello World!” plugin.
The first thing we’ll want to do is enumerate the features of the code which we need to test.
There are many ways you can approach this. My preference is to create a single test file per file in my source code and have my tests
directory mirror the structure of my source directory.
Each test file has tests for each function (or member functions of classes) in the corresponding source file.
So, the directory structure for our plugin would look like:
hello-world/
hello-world.php
phpunit.xml
tests/
bootstrap.php
test-hello-world.php
We’ll be doing our work in test-hello-world.php
where we’ll set up the skeleton of our test case, which is as simple as extending WP_UnitTestCase
:
<?php
class HelloWorldTest extends WP_UnitTestCase {}
We can then stub out test functions for each function:
class HelloWorldTest extends WP_UnitTestCase { function test_hello_world() { // Test hello_world() } function test_hello_world_markup() { // Test hello_world_markup() } function test_hello_world_css() { // Test hello_world_css() } }
Now, let's look at each function and consider what we're testing:
1. For hello_world
, we want to verify that the string returned is "Hello World!":
function test_hello_world() {
$this->assertEquals("Hello World!" == hello_world());
}
Easy enough. Now if for some reason someone changes the return value to "Hello world!" -- capitalization be damned -- the test will fail.
I can hear you now, "This is stupid, no one writes code like this." Yes, the example is stupid and that's the point. Don't get caught up focusing on the wrong thing.
It makes no difference how complex the function is, the only concern is verifying that the return value or the outcome is what is expected.
So, if instead you're testing some_made_up_function
which returns an object, you may want to verify that it is actually a PHP Object:
$this->assertEquals(gettype(some_made_up_function()), "object");
Or that the object has a specific member attribute:
$test_obj = some_made_up_function();
$this->assertTrue(property_exists($test_obj, 'name_of_attribute_goes_here'));
2. For hello_world_markup
, we want to verify that the function prints "<p id='hello-world'>Hello World!</p>":
function test_hello_world_markup() {
$this->expectOutputString("<p id='hello-world'>Hello World!</p>");
hello_world_markup();
}
Notice that we're expecting "Hello World!" to be part of the output. This might not be a good thing. If the return value of hello_world
changes, this test will fail, too.
For the sake of example, let's say we only care about testing the markup and not the message. We can take this test a step further and decouple the two so that we're only testing the markup by changing it to read:
function test_hello_world_markup() {
ob_start();
hello_world_markup();
$result = ob_get_contents();
ob_end_clean();
$this->assertTrue(!empty(preg_match("/^<p\s+id='hello-world'>.*<\/p>$/", $result)));
}
Essentially what we're saying is, we don't care what the message is as long as it is wrapped in <p id="hello-world" />
tag.
Simple, right?
3. For hello_world_css
, we want to verify that the function prints the CSS rules for our "Hello World!" markup:
function test_hello_world_css() {
ob_start();
hello_world_css();
$result = ob_get_contents();
ob_end_clean();
// Make sure there are style tags being printed. Duh.
$this->assertTrue(!empty(preg_match("/.*?
An example of a individual power player embed.
They consist of:
- Home/landing page
- Individual state page
- Individual power player page
- An embeddable version of the individual state page
- An embeddable version of the individual power player page
One of the goals for us was to create a resource that any of our member organizations could use to bolster their coverage of the elections — either by hosting information for an entire state, or including individual power player cards in articles covering campaign finance and the election.
Thus the embeddable versions of the state and power player pages. These are essentially the same as the normal templates, with a simple full-width layout and simplified INN branding (a credit link at the bottom of each embed).
The Kentucky Center for Investigative Reporting is one INN member making great use of the app. Here is a complete list of all the members organizations participating in the project and the stories they've published (so far).
The entire project is responsive (thanks to pym.js, yet another NPR project), made to look great no matter what size container the embeddable elements get placed in.
Snags and snafus
In working with the NPR app template we encountered some things that aren’t well documented (yet).
CSS and JS pseudo-tags
Tyler covers this briefly in another blog post on the app template.
What wasn’t clear at first was how the JS and CSS pseudo-tags interact across templates.
We ran into a problem where, in separate templates, we “pushed” or queued different sets of javascript files to be rendered. In both templates, we were passing the same file name to the render function, which resulted in the second file overwriting the first when deploying.
Here’s what NOT to do:
{# Template file no. 1 #}
{{ JS.push('js/lib/jquery.js') }}
{{ JS.push('js/lib/underscore.js') }}
{{ JS.push('js/lib/bootstrap.js') }}
{{ JS.push('js/lib/template_1.js') }}
{{ JS.render('js/app-footer.min.js') }}
{# Template file no. 2 #}
{{ JS.push('js/lib/jquery.js') }}
{{ JS.push('js/lib/bootstrap.js') }}
{{ JS.push('js/lib/template_2.js') }}
{{ JS.render('js/app-footer.min.js') }}
Once you realize that JS.render
outputs a file, the contents of which are determined by preceding calls to JS.push
, you realize that having different calls to JS.push
before rendering to the same file just won’t work.
In this case, if template 2 is rendered after template 1, “js/app-footer.min.js” will be missing “underscore.js”, potentially breaking functionality in template 1.
Do this instead:
{# Template file no. 1 #}
{{ JS.push('js/lib/jquery.js') }}
{{ JS.push('js/lib/underscore.js') }}
{{ JS.push('js/lib/bootstrap.js') }}
{{ JS.push('js/lib/template_1.js') }}
{{ JS.render('js/app-footer-1.min.js') }}
{# Template file no. 2 #}
{{ JS.push('js/lib/jquery.js') }}
{{ JS.push('js/lib/bootstrap.js') }}
{{ JS.push('js/lib/template_2.js') }}
{{ JS.render('js/app-footer-2.min.js') }}
By making the filename passed to JS.render
unique to each template, we can be sure we’re not clobbering any javascript files.
Flask’s url_for function and your project’s path prefix
Another issue we encountered was that the app template, using Flask’s default url_for
function, doesn’t take into consideration your project’s path. That is, when you deploy your app to S3, it is meant to live at something like http://apps.yourdomainname.org/project-slug/ whereas the development server uses something like http://localhost:8000/ without the project slug.
For example:
<a href="{{ url_for('some_view') }}">Hey, a link to a page</a>
Renders as:
<a href="/some-view/">Hey, a link to a page</a>
What we want when deploying to staging or production is an URL that includes the project slug:
<a href="/project-slug/some-view/">Hey, a link to a page</a>
To remedy this, we created an app_template_url_for
function to replace Flask’s standard url_for
. The app_template_url_for
figures out the current target environment (i.e. development, staging or production) and inserts the project slug as necessary.
View the source code here and here.
Another change we made to INN’s version of the app template is modifying the Flask app’s static_folder:
app = Flask(__name__, static_folder='www/assets')
View this change in context here.
What this does is allow you to use url_for
to build urls for static assets kept in www/assets.
<link rel="shortcut icon" href="{{ url_for("static", filename="icons/favicon.ico") }}" />
This provides the flexibility to include assets outside of the CSS and JS pseudo-tag framework if you find yourself with the need or desire to do so.
Working with Member Organizations
The Power Players project presented other challenges, too, because of the number and diversity of INN member newsrooms participating.
First, the data needed to be analyzed and populated in the app template for about a dozen states. Campaign finance data is notoriously dirty (especially when trying to look at individual donors, as we were), plus we were combining data from different sources, so the cleaning and analyzing stage took longer than expected. Some member organizations also wanted more custom analysis, such as looking at local data only rather than statewide data as most members were. This forced us to make some compromises in the design of the app like using the state outline for California but labelling it “San Diego.”
A project such as this also takes a great deal of communication. INN’s director of data services, Denise Malan, held a kick-off conference call with members and stayed in contact with them as she was analyzing the data and the tech team was designing and building the app. We also provided members with instructions on how to add photos and customized content to the app while it was still a work in progress.
As is the case with all our editorial collaborations, INN members can publish stories according to their own timetables, and several ran their stories before the app was finished because we were working right up to the deadline on Election Day. Others have yet to publish because they choose to wait for updated data with post-election numbers.
Ideally, we would love for all members participating in a project to use our work, and we learned some valuable lessons in the process of building our first editorial app.
In future projects, we would like to have mockups of the news app ready as early as possible so our members are able to visualize what we will be providing and how this will fit with their reporting. We also want to provide a firmer deadline for launching an app so members can plan their publishing dates accordingly and leave time to implement the stuff we build. We’ll also do a better job providing the partners with documentation to make it as easy as possible for them to adopt our work.
Conclusion
We learned a lot in the process of building our first app, both about the NPR app template and also what it takes to manage a complex project working with so many partner organizations.
Will we use our fork of the NPR app template for everything? Probably not. We’ll continue to experiment and try out different things before settling on our default set of tools and templates. For projects where it’s a good fit or where we need to deploy something quick and easy, we definitely plan to use it as a solid starting point in building future apps.
Since this is our first app and we’re still learning, we’d love to hear your thoughts and feedback. You can find our team on Twitter @INNnerds or send us email at nerds@inn.org.