Fragmented Thought
TwitterGitHub

Automated End To End Testing for Angular

By

Published:

Lance Gliser

Heads up! This content is more than six months old. Take some time to verify everything still works as expected.

I first learned protractor for use in End to End (E2E) testing for bigshotbracket.com. That was a savage crash course, with a lot of mistakes. My latest project has given me a chance to review how I implemented that testing, and improve on it. I finally feel like I understand it well enough to share with confidence that I'm doing things correctly. So I'll expand on the concepts from the official tutorial and its limited handling of page abstractions.

For reference, I'm building an application that uses the emfluence Marketing Platform. It creates a specialized front end, that simplifies steps and adds business logic specific to a client to produce something faster and more efficient than a generic platform could ever achieve. The tests you'll see here are specific to the html, angular code, and functionality you'd see in that application. I'm providing this context to ensure the tests make sense. We're going to be using a third party API to create and send email, with all the lovely steps in between. You'll always have to adapt testing to your own application.

Getting Started

To begin, you'll need npm, java, and protractor. Installing npm and java are out of the scope of this article, but here are the commands for protractor. From the command line run:

// Install protractor globally (it's kind of large)
sudo npm install -g protractor
// Install web-drivers as required (these are why it is large)
sudo webdriver-manager update

Before you can begin running tests, you'll need selenimum running:

webdriver-manager start

Then when you're ready, you can fire off a test suite using the following command from the root, if you used the directory setup in this tutorial:

protractor protractor/conf.js --suite {name}

Directory Structure

httpdocs/
-- protractor/ (e2e test directory)
-- -- pages/ (page definitions)
-- -- -- authentication/
-- -- -- -- logout.page.js
-- -- -- -- reload.page.js
-- -- -- -- token.page.js
-- -- -- emails/
-- -- -- -- edit.page.js
-- -- -- -- index.page.js
-- -- -- -- new.page.js
-- -- -- base.page.js
-- -- -- style-guide.page.js
-- -- specs/ (actual tests)
-- -- -- authentication/
-- -- -- -- failure.spec.js
-- -- -- -- load-from-access-token.spec.js
-- -- -- -- logout.spec.js
-- -- -- emails/
-- -- -- -- delete-draft.spec.js
-- -- -- -- index.spec.js
-- -- -- -- new.spec.js
-- -- -- style-guide/
-- -- -- -- spec.js
-- -- conf.js (protractor configuration file)
-- -- environment.js (configuration files specific to the webhost)
-- index.html (main angular index.html file)

Breakdown

I'll touch briefly on what each file does. Conf.js and environment.js are the most important. After that, they are more just useful for seeing how I approach finding, and testing conditions. For reference, I've taken a shortcut on authorization through the marketing platform itself. All off the tests here use a 'fake' login step I reproduced just for testing. By providing the environment.js file with an access token from your session, we bypass that tricky piece just for speed of building tests. It's a slight OAuth security hole, I know. It's worth it to not build tests for a third party that already tests itself.

conf.js

protractor/conf.js

This file is the main definition and starting point for all protractor tests. We use it define globals, variables, organize tests into suits, and more. I tend to include directions for other developers at the top of this file.

// To install protactor:
// sudo npm install -g protractor
// sudo webdriver-manager update

// To run protactor tests use:
// Ensure selenimum is running
// webdriver-manager start
// Fire off a test suite
// protractor protractor/conf.js --suite name
// Or protractor debug protractor/conf.js --suite name (No idea how this works yet)

// Note that $ and $$ are not jQuery
// $: http://angular.github.io/protractor/#/api?view=build$
// $$: http://angular.github.io/protractor/#/api?view=build$$
// Full api documentation is at: angular.github.io/protractor/#/api
// Full expect() and toBe documenation is at: http://jasmine.github.io/2.1/introduction.html

var env = require('./environment.js');

// **dirname retuns a path of this particular config file
var basePath = **dirname + '/';

exports.config = {
// Location of ng-app tag in the root index.html file
// rootElement: 'html',

// The address of a running selenium server.
seleniumAddress: env.seleniumAddress,

// Capabilities to be passed to the webdriver instance.
capabilities: env.capabilities,

// Spec patterns are relative to the location of the spec file. They may include glob patterns:
// {directory}/\*.js is a one level wildcard
// {directory}/\*\*.js is a multilevel wildcard
// I generally prefer to run my tests in a defined order, as they may depend on actions from the previous
suites: {
styles: [ 'specs/style-guide/spec.min.js' ],
// Potential touch point to launch accessibility testing available through additional chrome tools
accessibility: [ 'specs/style-guide/spec.min.js' ],
authentication: [
'specs/authentication/failure.spec.js',
'specs/authentication/load-from-access-token.spec.js',
'specs/authentication/logout.spec.js',
],
emails: [
// We have to log in before using the next tests
'specs/authentication/load-from-access-token.spec.js',
// Now we run the specs in the order required to not leave too much trash
// General index
'specs/emails/index.spec.js',
// Create and delete a draft
'specs/emails/new.spec.js',
'specs/emails/delete-draft.spec.js',
// TODO Create, add contacts, schedule, cancel schedule, and send
'specs/emails/new-email.spec.js',
'specs/emails/edit.spec.js',
'specs/emails/contacts-selection.spec.js', // TODO
'specs/emails/schedule.spec.js', // TODO
'specs/emails/cancel-schedule.spec.js', // TODO
'specs/emails/send.spec.js', // TODO
],
},

// Base url
baseUrl: env.baseUrl,

// Options to be passed to Jasmine-node.
jasmineNodeOpts: {
showColors: true, // Use colors in the command line report.
},

/_
plugins: [{
// Accessibility plugin setup
chromeA11YDevTools: true,
path: '/opt/local/lib/node_modules/protractor/plugins/accessibility'
}]
_/

onPrepare: function () {
// "relativePath" - path, relative to "basePath" variable
global.requirePage = function (relativePath) {
return require(basePath + 'pages/' + relativePath + '.page.min.js');
};

    global.getErrors = function(){
      return $$('.alert-box.alert');
    };
    global.getMessages = function(){
      return $$('.alert-box.success');
    };

    global.escapeRegExp = function(str) {
      return str.replace(/[\-\[\]\/\{\}\(\)\*\+\?\.\\\^\$\|]/g, "\\$&");
    }

}

};

// These will be made available in browser.params inside tests
exports.config.params = env.params;
exports.config.params.currentUrl = undefined;
exports.config.params.testEmail = {
title: 'Protractor Test Email'
};

environment.js

protractor/environment.js

This file is the environment specific definitions. We use it define items need in local or dev environments. This file should generally be in your .gitignore.

module.exports = {
  // The address of a running selenium server.
  seleniumAddress: process.env.SELENIUM_URL || "http://localhost:4444/wd/hub",

  // Capabilities to be passed to the webdriver instance.
  capabilities: {
    browserName: process.env.TEST_BROWSER_NAME || "chrome",
    version: process.env.TEST_BROWSER_VERSION || "ANY",
  },

  baseUrl: "https://local.example.com/",

  // These will be made available in browser.params inside tests
  params: {
    accessToken: "{some-value}",
  },
};

spec.js

protractor/specs/style-guide/spec.js

This a basic spec definition file. It includes one or more 'definition' and 'it' blocks. These represent logic groupings of functional tests.

// Define the 'Page' object by using the global.requirePage function from conf.js to require it
var StyleGuidePage = requirePage("style-guide");

// Create a new instance of the 'Page' object
var page = new StyleGuidePage();

// Starting describing the groupings
describe("basic element definitions", function () {
  // Create a test using 'it'.
  // 'it' functions run asynchronously, but in queue order for each other
  it("should define h1", function () {
    // Note that page.get() _must_ called inside the first 'it' function
    page.get();
    expect(element(by.tagName("h1")).getText()).toBeDefined();
  });
  it("should define h2", function () {
    expect(element(by.tagName("h2")).getText()).toBeDefined();
  });
  it("should define h3", function () {
    expect(element(by.tagName("h3")).getText()).toBeDefined();
  });
  it("should define h4", function () {
    expect(element(by.tagName("h4")).getText()).toBeDefined();
  });
  it("should define h5", function () {
    expect(element(by.tagName("h5")).getText()).toBeDefined();
  });
  it("should define h6", function () {
    expect(element(by.tagName("h6")).getText()).toBeDefined();
  });
  it("should define tables", function () {
    expect(element(by.tagName("table")).getText()).toBeDefined();
  });
});

describe("basic message handling", function () {
  // Get the errors using the global.getErrors function from conf.js
  var errors = getErrors();
  // errors was selected using \$\$. It returns a group (list), so you'll have to check pieces
  it("should generate error messages", function () {
    expect(errors).toBeDefined();
    expect(errors.count()).toBe(1);
  });
  it('error messages should contain "sample error"', function () {
    expect(errors.get(0).getText()).toContain("Sample error");
  });
  // Get the messages using the global.getMessages function from conf.js
  var messages = getMessages();
  it("should generate messages", function () {
    expect(messages).toBeDefined();
    expect(messages.count()).toBe(1);
  });
  it('messages should contain "sample message"', function () {
    expect(messages.get(0).getText()).toContain("Sample message");
  });
});

// Special handling for unique elements such as 'Hold to activate' buttons
describe("hold buttons", function () {
  var holdButton = page.getHoldButton();
  it("should give feedback on interaction", function () {
    browser.actions().mouseDown(holdButton).perform();
    browser.sleep(50);
    expect(holdButton.getAttribute("class")).toMatch("holding");
    browser.actions().mouseUp(holdButton).perform();
  });
  it("should not complete in under .5 seconds", function () {
    browser.actions().mouseDown(holdButton).perform();
    browser.sleep(50);
    browser.actions().mouseUp(holdButton).perform();
    expect(holdButton.getAttribute("class")).toMatch("hold-failure");
  });
  it("should complete eventually", function () {
    browser.actions().mouseDown(holdButton).perform();
    browser.sleep(2000);
    browser.actions().mouseUp(holdButton).perform();
    expect(holdButton.getAttribute("class")).toMatch("hold-success");
  });
});

base.page.js

protractor/pages/base.page.js

The base page serves as a way to collect standard functionality that can later be inherited in other pages. These methods will be used by the style-guide.page.js file below.

var BasePage = function () {};

BasePage.prototype.path = "#/";

BasePage.prototype.getPath = function () {
  return path;
};

// This is used in page to page reloads. You'll see more details on usage later.
BasePage.prototype.getPathRegex = function () {
  var regex = escapeRegExp(this.getPath());
  return new RegExp(regex);
};

/**
 * - Gets the page directly by loading a url. This will not reload the current page
 */
BasePage.prototype.get = function () {
  var url = browser.baseUrl + this.getPath();

  if (browser.params.currentUrl !== url) {
    browser.params.currentUrl = url;
    return browser.get(url);
  } else {
    return true;
  }
};

module.exports = BasePage;

style-guide.page.js

protractor/pages/style-guide.page.js

The previous spec file referenced page level functions frequently. The logic to find these elements and page variables are contained in page.js files to keep logic simple to maintain. See the protractor page object reference for more on the concept of why you need this.

This a basic spec definition file. It includes one or more 'definition' and 'it' blocks. These represent logic groupings of functional tests.

"use strict";

var BasePage = requirePage("base");

var StyleGuidePage = function () {};

StyleGuidePage.prototype = new BasePage();

StyleGuidePage.prototype.path = "style-guide";

StyleGuidePage.prototype.getLoginLink = function () {
  return element(by.id("header-login"));
};

StyleGuidePage.prototype.getHoldButton = function () {
  return $("button[hold-button]");
};

module.exports = StyleGuidePage;

Additional Tests and Pages

Just to show some additional functionality to adapt from, and how I've handled page to page testing with the least duplication possible, here's the 'create new email that redirects to editing email' test. Note that I'm heavily reliant here on Protractor's natural handling. Any action that kicks off an angular action such as $http, causes testing to stop until Angular resolves.

new.spec.js

protractor/specs/email/new.page.js

var NewEmailPage = requirePage('emails/new');

// Note that page.get HAS to be used inside an it() call.
var page = new NewEmailPage();

describe('new email page', function() {
    var titleInput, firstNameInput, templates, templateThumbnails, continueButton;

    it('should contain a required title input', function () {
        page.get();
        titleInput = page.getTitleInput();
        expect(titleInput.isPresent()).toBeTruthy();
        expect(titleInput.getAttribute('required')).toBeTruthy();
    });

    it('should contain a required from name input', function () {
        firstNameInput = page.getFromNameInput();
        expect(firstNameInput.isPresent()).toBeTruthy();
        expect(firstNameInput.getAttribute('required')).toBeTruthy();
    });

    it('should default your from name based on your account', function(){
        expect(firstNameInput.getAttribute('value')).toBeTruthy();
    });

    it('should contain a list of templates available to you', function () {
        templates = page.getTemplates();
        expect(templates.count()).toBeGreaterThan(0);
    });

    it('should set the templateId model when a thumbnail is clicked', function () {
        templateThumbnails = page.getTemplateThumbnails();
        templateThumbnails.get(0).click();
        expect(templates.get(0).getAttribute('checked')).toBeTruthy();
    });

    it('should enable the next button only after filling in title, from, and template', function () {
        continueButton = page.getContinueButton();
        expect(continueButton.getAttribute('disabled')).toBeTruthy();
        titleInput.sendKeys( browser.params.testEmail.title );
        templateThumbnails.get(0).click();
        expect(continueButton.getAttribute('disabled')).not.toBeTruthy();
    });

    it('should redirect you to the edit step when you click continue', function () {
        var nextPage = page.getNextPage();
        continueButton.click().then(function(){
        var urlPromise = browser.getCurrentUrl();
        expect( urlPromise ).toMatch( nextPage.getPathRegex());
        urlPromise.then(function(url){
        // Make sure to alert the BasePage we've moved
        browser.params.currentUrl = url;
    });
});

new.page.js

protractor/pages/email/new.page.js

"use strict";

var BasePage = requirePage("base");
var EditEmailPage = requirePage("emails/edit");

var NewEmailPage = function () {};

NewEmailPage.prototype = new BasePage();

NewEmailPage.prototype.path = "emails/new";

NewEmailPage.prototype.getTitleInput = function () {
  return element(by.model("title"));
};

NewEmailPage.prototype.getFromNameInput = function () {
  return element(by.model("fromName"));
};

NewEmailPage.prototype.getTemplates = function () {
  return $$("ul.templates li input[type=radio]");
};

NewEmailPage.prototype.getTemplateThumbnails = function () {
  return $$("ul.templates li label img");
};

NewEmailPage.prototype.getContinueButton = function () {
  return element(by.id("continue"));
};

EditEmailPage.prototype.getNextPage = function () {
  return new EditEmailPage();
};

module.exports = NewEmailPage;
<h4>edit.page.js</h4> <pre>protractor/pages/email/edit.page.js</pre> ```js 'use strict';

var BasePage = requirePage('Base');

var EditEmailPage = function(){};

EditEmailPage.prototype = new BasePage();

EditEmailPage.prototype.path = 'emails/#';

EditEmailPage.prototype.getTitleInput = function(){ return element(by.model('title')); };

EditEmailPage.prototype.getFromNameInput = function(){ return element(by.model('fromName')); };

EditEmailPage.prototype.getContinueButton = function(){ return element(by.id('continue')); };

module.exports = EditEmailPage;


There are still a load of things I'm learning, mostly about DOM
selection, and the limitations of Jasmine's `expect()` options.
It's getting clearer, and I'm really looking forward to being
able to ensure the entire application works as expected in under
30 seconds.

One last little bonus, as it took me a while to figure it out:
Here's how to find a row with a link with a class of 'title' and
text of 'Protractor Test Email', then return the parent row, and
eventually the delete button:

```js
// The draft list. A nice simple id
var drafts = element( by.id('drafts-list') );
// The row, with the class and title match, and back up to the row
var email = drafts.element(by.cssContainingText('.title', title)).element(by.xpath('ancestor::tr'));
// And the child delete button
var button = email.\$('button.delete');

Now there's a good intro to how you can find your element...

Some gotchas

Having spent the last two weeks deep in testing, I thought I'd share findings with you:

  • Always call browser.driver.manage().window().setSize() and provide random X and Y to simulate different devices during your conf.js onPrepare.
  • You can and should test memory allocation when testing in chrome. Use browser.driver.executeScript('return window.performance.memory.usedJSHeapSize;').then(function(heapSize){ });
  • browser.driver.executeScript('something()', element) will report "Failed: Maximum call stack size exceeded" if the argument you're passing in is too large. Always pass in using element.getWebElement() instead.
  • browser.driver.executeScript('something()', element) will report "Failed: Maximum call stack size exceeded" if the return value is too large. Return only what you really need.