pixel

Unit Testing Alexa Skills

We have been doing a great deal of work with the new AudioPlayer interface.

As part of this work, we recognized the need to be able to automatically unit test skills that we develop for Alexa, both during development as well as part of Continuous Integration.

This has always been a need - but with the release of the AudioPlayer, with its additional complexity, it has become essential.

We have previously released our bst speak and bst intend commands.

We found these to be very useful for doing basic debugging and manual-testing.

But now for unit tests and CI, we have expanded on this, building a full-fledged Alexa emulator. It is meant to be interacted with programmatically - to create automated unit tests and integrate neatly into a CI process.

Why We Think This Is So Useful

Up until now, ensuring skill quality has been done via a combination of several, not-so-reliable techniques:

  • + Testing with a real Alexa device (i.e., an Echo - assuming you are lucky enough to live in a country where it is available for sale :-))
  • + Testing with the Alexa Service Simulator
  • + Testing with EchoSim.io (which does not support the AudioPlayer interface)
  • + Testing with hand-rolled JSON requests

Putting all of these together, one can gain a degree of confidence in a skill.

However, with each change to the code, these processes need to be repeated. Even if the QA steps are documented and executed perfectly, it is still easy to miss something.

And for many skills, these steps are likely not documented at all. This creates serious regression testing challenges.

As developers, we luckily came up with an answer to this some time ago - automated unit tests. It just has not been readily available for the Alexa until now.

The Bespoken Tools API

The API can be found here.

In particular, the main class used for testing is BSTAlexa.

Though it is written in JavaScript, it can be used to test Skills written in other languages. This is because the emulator communicates with a skill over HTTP.

So tests can be written in Javascript to test skills written in Python, Java or whatever other language is preferred. Example of this will be coming soon.

How It Works

Here is a quick sample of what a unit test looks like written using our emulator:

it('Launches and then plays first', function (done) {
    // Launch the skill via sending it a LaunchRequest
    alexa.launched(function (error, payload) {
        // Check that the introduction is played as outputSpeech
        assert.equal(payload.response.outputSpeech.ssml,
            '<speak> <audio src="https://s3.amazonaws.com/bespoken/streaming/bespokenspodcast-INTRODUCTION.mp3" />' +
            'You can say play, scan titles, or about the podcast </speak>');

        // Emulate the user saying 'Play'
        alexa.spoken('Play', function (error, payload) {
            // Ensure the correct directive and audioItem is returned
            assert.equal(payload.response.directives[0].type, 'AudioPlayer.Play');
            assert.equal(payload.response.directives[0].audioItem.stream.token, '0');
            assert.equal(payload.response.directives[0].audioItem.stream.url,
                'https://traffic.libsyn.com/bespoken/TIP103.mp3?dest-id=432208');
            done();
        });
    });
});

We have created a detailed tutorial here. It walks you through getting setup with the emulator and provides more detailed information on how to use it.

Additionally, we have created a reasonably complex sample project that demonstrates the AudioPlayer interface and the emulator working together.

We call it the Streamer. It takes a Podcast RSSFeed and delivers it to Alexa. It’s a neat use of the AudioPlayer interface and we encourage you to check it out.

The unit tests using the BSTAlexa emulator can be found here.

Additionally, we have in turn hooked it into a CI framework. Codeship, in this case, because of its excellent native support for Docker (we use Docker because our skill relies on ffmpeg - a non-trivial dependency to setup in a production/runtime environment).

However, ffmpeg aside, any CI framework can be used with it.

What Is Next?

We would love to get feedback on our API. Though it covers many aspects of the Alexa service, there are additional parts we plan to add, including:

The Standard Stuff

Talk to us on Gitter.

Stay updated either through GitHub:

Develop Alexa Skills Faster

Learn More

Subscribe

Get the next post delivered to you!

Recent