REST API Testing with Flagpole (Part 2)

Now that you got the hang of it with the previous blog post, let’s dive into more advanced techniques.

In my previous post, we went over getting started with automated REST API tests in Flagpole. It took you through the basics of installing the CLI, getting the package added to your application, and writing a few basic tests. You probably want to read that first, if you haven’t already.

Now that we nailed down the simple stuff, let’s dive a little deeper and help you build out some more robust tests. We’ll also tease out a little bit of how Flagpole can be used for things other than just REST APIs.

For most of the examples here, we’ll just run the test against the first result in the API call from the previous article. In that post, we used the iTunes API to pull all the results for “2pac” of type music videos. Here a gist I created with the iTunes Search API JSON output.

So we pluck out the first result with this:

let firstResult = response.select('results').first();

Now we expect that our results returned for 2pac are going to of the Rap or Hip-Hop genre. However, let’s say we’re not exactly sure what Apple is going to call it… so we’re going to test it against a regular expression.

firstResult.property('primaryGenreName').matches(/hip-?hop/i);

Assuming you understand regular expressions, the matches(pattern) method is dead simple. So let’s move on…

Next up, we want to be sure that the price to purchase the track is within some sane range. So let’s introduce the between(min, max) method.

firstResult
    .label('Track Price should be between $0 and $10')
    .property('trackPrice').between(0, 10);

Notice that we used the label method before it. If we didn’t do this it would be fine. However, this lets us set a really human readable assertion message.

I cleared out the assertions from the last article. So with only the code above in our assertions block, when we run our test the output should look like this:

The iTunes API also gives us the track’s duration in milliseconds. We don’t know exactly what range that it will be in, but we’re sure that it should be a number. So let’s test for the type of that field.

firstResult.property('trackTimeMillis').is('number');

If that field is a string or an object or null or anything else, besides a number, it would fail. So obviously you could also use ‘string’ or ‘array’ or whatever in that is(type) method.

The API also has a field called trackExplicitness, which can only be one of two values. So we want to make sure it’s one of those.

firstResult.property('trackExplicitness').in(['explicit', 'notExplicit']);

That’s all there is to it. The in(enum) method takes an array of possible values. For strings, it is case-sensitive. So if you wanted to make it fuzzy case matching you could do this:

firstResult.property('trackExplicitness').toLowerCase().in(['explicit', 'notexplicit']);

Sometimes there are things that we want to test for that Flagpole may not have specifically implemented. That’s okay! You can bring your own testing logic or even bring in additional validation libraries to make assertions.

In this case, the iTunes API has a property called releaseDate that we know should be in a valid date format. It will actually look like “1996–05–10T07:00:00Z” and we want to make sure it will parse as a real date.

Since Flagpole doesn’t have a date validation method currently, we will just do our own test with the assert(bool, message) method.

firstResult.assert(
    Date.parse(firstResult.property('releaseDate')) !== NaN,
    'Release Date is a valid date'
);

All we did there was use the native JavaScript Date.parse method. This takes a string argument. If it’s in a recognized date format, it returns the unix timestamp (which would be an integer). If it’s an invalid date format, it would return NaN (not a number).

The first argument to our assert method should evaluate true or false. The second argument is the pass/fail message, so should make it something super readable to describe what we are testing for.

Here’s a check in of what our test output should look like:

Okay, now that we’re done with the easy stuff... We don’t want to test just one single end point. We could obviously add additional scenarios to test other API end points or different query string parameters. And you should!

But that would just be as simple as making a new scenario the same way we made this one. And hopefully you can figure that out. Within this API result set though, it has links to other pages.

Specifically, we notice that there are links to the artwork for each music video. Any thorough QA automation engineer would want to make sure that those URLs are valid links and they actually resolve and don’t 404. So let’s test it…

firstResult.property('artworkUrl30')
    .load('Verify Image', true);

The artworkUrl30 field has a URL of a tiny thumbnail version of the music video. Actually I’m not sure what you can even use it for it’s so tiny, but I digress… regardless, above we grab the property with the URL and we tell Flagpole to load it.

What this is actually doing is creating a new Scenario on the fly. I think of what we are doing here (and proceeding examples) as kind of a lambda scenario. Normally we’d have to add the assertions for this new on-the-fly scenario; however, setting the second (optional) argument to true tells Flagpole to just infer the assertions. It will execute immediately.

Whoa! So it automatically generated that scenario for us. It tested that it loaded and it even tested that the mime type matched that of an image. But we didn’t even tell it that it was an image!

In all cases, Flagpole does it best to assume what you don’t explicitly tell it. In later tutorials, you’ll see that it knows a URL found inside an <img src> tag will be an image and <link rel=”stylesheet” href=”….”> will be a Stylesheet. But it also looks at the URL itself to guess. So it sees the .jpg at the end of the URL so it will infer that it should treat it like an image.

If we want to do additional assertions to run more tests we can do it like this:

firstResult.property('artworkUrl30').load('Verify Image')
    .image()
    .assertions(function (image) {
        image.select('type').equals('jpg');
        image.select('width').equals(30);
        image.select('height').lessThanOrEquals(30);
    });

Notice here, we specifically told it with .image() to treat this scenario as an image response. We wouldn’t need to since the URL ended in .jpg, but just know that you can override this inferred setting or just use it to be certain that Flagpole does the right thing.

We added additional assertions that the file was in jpg format. The “30” in the artworkUrl30 property indicates that it should be 30 pixels wide. So we can add a test for that to make sure it’s returning the right thumbnail. And for good measure we expect the height to be less than or equal to the width.

There is also a field called artistViewUrl that leads to an HTML page highlighting all of the artist’s work. It looks like this:

So we want to verify that this URL is correct. And that it leads to a page with the right content on it. No problem! Very similar to testing the image above, we can create a new lambda scenario for that HTML page.

firstResult.property('artistViewUrl').load('Artist Page')
    .html()
    .assertions(function (page) {
        page.select('title').toLowerCase().contains('2pac');
        page.select('h1.page-header__title')
            .label('Main page heading has artist name')
            .similarTo('2pac');
        page.select('.we-shelf-songs li').length()
            .label('Six items in the top songs list')
            .equals(6);
        page.select('.we-lockup__title div').first()
            .text().similarTo('Me Against the World');
    });

Above we are loading the artist view URL, telling Flagpole to interpret it as an HTML page. We add assertions that:

• Make sure the title includes “2pac”.
• Make sure the main page header also has “2pac”.
• Look for the list of top songs, which should always show six songs.
• Look for the “Essential Album” section (as it is labeled on the Apple Music 2pac page) and that this section shows “Me Against the Word” as the top album.

We could add all kinds of other tests for that HTML page, but I’ll do a follow up blog post to focus more on HTML test automation.

Our entire output now looks like this:

What’s cool is with little to no more code, we could actually test every single entry on this page with an each loop. So much we can do with so little code!

Here is the source code for this blog post.

Well that’s a rap for now. (Get it? 2pac? Rap? …… anyway) Hopefully this is helpful and will kick start some new adoption of what I hope you agree is a promising QA framework in its early stages. If you guys are into it, I’ll keep rolling these out. There are plenty of more topics to dive into with both how to write tests, use Flagpole, and how to integrate this automation into your stack and QA work flow.

Let me know your thoughts and questions in the comments!