Automated End To End Testing for Angular

I first learned protractor for use in end to end (E2E) testing for bigshotbracket.com. That was a savage crash course, with a lot of mistakes. My latest project has given me a chance to review how I implemented that testing, and improve on it. I finally feel like I understand it well enough to share with confidence that I'm doing things correctly. So I'll expand on the concepts from the official tutorial and it's limited handling of page abstractions.

For reference, I'm building an application that uses the emfluence Marketing Platform. It creates a specialized front end, that simplifies steps and adds business logic specific to a client to produce something faster and more efficient than a generic platform could ever achieve. The tests you'll see here are specific to the html, angular code, and functionality you'd see in that application. I'm providing this context to ensure the tests make sense. We're going to be using a third party API to create and send email, with all the lovely steps in between. You'll always have to adapt testing to your own application.

Getting Started

To begin, you'll need npm, java, and protractor. Installing npm and java are out of the scope of this article, but here are the commands for protractor. From the command line run:

  1. // Install protractor globally (it's kind of large)
  2. sudo npm install -g protractor
  3. // Install web-drivers as required (these are why it is large)
  4. sudo webdriver-manager update

Before you can begin running tests, you'll need selenimum running:

  1. webdriver-manager start

Then when you're ready, you can fire off a test suite using the following command from the root, if you used the directory setup in this tutorial:

  1. protractor protractor/conf.js --suite {name}

Directory Structure

httpdocs/
-- protractor/ (e2e test directory)
-- -- pages/ (page definitions)
-- -- -- authentication/
-- -- -- -- logout.page.js
-- -- -- -- reload.page.js
-- -- -- -- token.page.js
-- -- -- emails/
-- -- -- -- edit.page.js
-- -- -- -- index.page.js
-- -- -- -- new.page.js
-- -- -- base.page.js
-- -- -- style-guide.page.js
-- -- specs/ (actual tests)
-- -- -- authentication/
-- -- -- -- failure.spec.js
-- -- -- -- load-from-access-token.spec.js
-- -- -- -- logout.spec.js
-- -- -- emails/
-- -- -- -- delete-draft.spec.js
-- -- -- -- index.spec.js
-- -- -- -- new.spec.js
-- -- -- style-guide/
-- -- -- -- spec.js
-- -- conf.js (protractor configuration file)
-- -- environment.js (configuration files specific to the webhost)
-- index.html (main angular index.html file)

Breakdown

I'll touch briefly on what each file does. Conf.js and environment.js are the most important. After that, they are more just useful for seeing how I approach finding, and testing conditions. For reference, I've taken a short cut on authorization through the marketing platform itself. All off the tests here use a 'fake' login step I reproduced just for testing. By providing the environment.js file with an access token from your session, we bypass that tricky piece just for speed of building tests. It's a slight OAuth security hole, I know. But it's worth it to not build tests for a third party that already tests itself.

conf.js

protractor/conf.js

This file is the main definition and starting point for all protractor tests. We use it define globals, variables, organize tests into suits, and more.
I tend to include directions for other developers at the top of this file.

  1. // To install protactor:
  2. // sudo npm install -g protractor
  3. // sudo webdriver-manager update
  4.  
  5. // To run protactor tests use:
  6. // Ensure selenimum is running
  7. // webdriver-manager start
  8. // Fire off a test suite
  9. // protractor protractor/conf.js --suite name
  10. // Or protractor debug protractor/conf.js --suite name (No idea how this works yet)
  11.  
  12. // Note that $ and $$ are not jQuery
  13. // Full api documentation is at: angular.github.io/protractor/#/api
  14. // Full expect() and toBe documenation is at: http://jasmine.github.io/2.1/introduction.html
  15.  
  16. var env = require('./environment.js');
  17.  
  18. // __dirname retuns a path of this particular config file
  19. var basePath = __dirname + '/';
  20.  
  21. exports.config = {
  22.   // Location of ng-app tag in the root index.html file
  23.   // rootElement: 'html',
  24.  
  25.   // The address of a running selenium server.
  26.   seleniumAddress: env.seleniumAddress,
  27.  
  28.   // Capabilities to be passed to the webdriver instance.
  29.   capabilities: env.capabilities,
  30.  
  31.   // Spec patterns are relative to the location of the spec file. They may include glob patterns:
  32.   //   {directory}/*.js is a one level wildcard
  33.   //   {directory}/**.js is a multilevel wildcard
  34.   // I generally prefer to run my tests in a defined order, as they may depend on actions from the previous
  35.   suites: {
  36.     styles: [ 'specs/style-guide/spec.min.js' ],
  37.     // Potential touch point to launch accessibility testing available through additional chrome tools
  38.     accessibility: [ 'specs/style-guide/spec.min.js' ],
  39.     authentication: [
  40.       'specs/authentication/failure.spec.js',
  41.       'specs/authentication/load-from-access-token.spec.js',
  42.       'specs/authentication/logout.spec.js',
  43.     ],
  44.     emails: [
  45.       // We have to log in before using the next tests
  46.       'specs/authentication/load-from-access-token.spec.js',
  47.       // Now we run the specs in the order required to not leave too much trash
  48.       // General index
  49.       'specs/emails/index.spec.js',
  50.       // Create and delete a draft
  51.       'specs/emails/new.spec.js',
  52.       'specs/emails/delete-draft.spec.js',
  53.       // TODO Create, add contacts, schedule, cancel schedule, and send
  54.       'specs/emails/new-email.spec.js',
  55.       'specs/emails/edit.spec.js',
  56.       'specs/emails/contacts-selection.spec.js', // TODO
  57.       'specs/emails/schedule.spec.js', // TODO
  58.       'specs/emails/cancel-schedule.spec.js', // TODO
  59.       'specs/emails/send.spec.js', // TODO
  60.     ],
  61.   },
  62.  
  63.   // Base url
  64.   baseUrl: env.baseUrl,
  65.  
  66.   // Options to be passed to Jasmine-node.
  67.   jasmineNodeOpts: {
  68.     showColors: true, // Use colors in the command line report.
  69.   },
  70.  
  71.   /*
  72.   plugins: [{
  73.     // Accessibility plugin setup
  74.     chromeA11YDevTools: true,
  75.     path: '/opt/local/lib/node_modules/protractor/plugins/accessibility'
  76.   }]
  77.   */
  78.  
  79.   onPrepare: function () {
  80.     // "relativePath" - path, relative to "basePath" variable
  81.     global.requirePage = function (relativePath) {
  82.       return require(basePath + 'pages/' + relativePath + '.page.min.js');
  83.     };
  84.  
  85.     global.getErrors = function(){
  86.       return $$('.alert-box.alert');
  87.     };
  88.     global.getMessages = function(){
  89.       return $$('.alert-box.success');
  90.     };
  91.  
  92.     global.escapeRegExp = function(str) {
  93.       return str.replace(/[\-\[\]\/\{\}\(\)\*\+\?\.\\\^\$\|]/g, "\\$&");
  94.     }
  95.   }
  96.  
  97. };
  98.  
  99. // These will be made available in browser.params inside tests
  100. exports.config.params = env.params;
  101. exports.config.params.currentUrl = undefined;
  102. exports.config.params.testEmail = {
  103.   title: 'Protractor Test Email'
  104. };

environment.js

protractor/environment.js

This file is the environment specific definitions. We use it define items need in local or dev environments. This file should generally be in your .gitignore.

  1. module.exports = {
  2.   // The address of a running selenium server.
  3.   seleniumAddress:
  4.     (process.env.SELENIUM_URL || 'http://localhost:4444/wd/hub'),
  5.  
  6.   // Capabilities to be passed to the webdriver instance.
  7.   capabilities: {
  8.     'browserName':
  9.       (process.env.TEST_BROWSER_NAME || 'chrome'),
  10.     'version':
  11.       (process.env.TEST_BROWSER_VERSION || 'ANY')
  12.   },
  13.  
  14.  
  15.   // These will be made available in browser.params inside tests
  16.   params: {
  17.     "accessToken": "{some-value}"
  18.   }
  19. };

spec.js

protractor/specs/style-guide/spec.js

This a basic spec definition file. It includes one or more 'definition' and 'it' blocks. These represent logic groupings of functional tests.

  1. // Define the 'Page' object by using the global.requirePage function from conf.js to require it
  2. var StyleGuidePage = requirePage('style-guide');
  3.  
  4. // Create a new instance of the 'Page' object
  5. var page = new StyleGuidePage();
  6.  
  7. // Starting describing the groupings
  8. describe('basic element definitions', function() {
  9.   // Create a test using 'it'.
  10.   // 'it' functions run asynchronously, but in queue order for each other
  11.   it('should define h1', function() {
  12.     // Note that page.get() *must* called inside the first 'it' function
  13.     page.get();
  14.     expect( element( by.tagName('h1')).getText() ).toBeDefined();
  15.   });
  16.   it('should define h2', function() {
  17.     expect( element( by.tagName('h2')).getText() ).toBeDefined();
  18.   });
  19.   it('should define h3', function() {
  20.     expect( element( by.tagName('h3')).getText() ).toBeDefined();
  21.   });
  22.   it('should define h4', function() {
  23.     expect( element( by.tagName('h4')).getText() ).toBeDefined();
  24.   });
  25.   it('should define h5', function() {
  26.     expect( element( by.tagName('h5')).getText() ).toBeDefined();
  27.   });
  28.   it('should define h6', function() {
  29.     expect( element( by.tagName('h6')).getText() ).toBeDefined();
  30.   });
  31.   it('should define tables', function() {
  32.     expect( element( by.tagName('table')).getText() ).toBeDefined();
  33.   });
  34. });
  35.  
  36. describe('basic message handling', function() {
  37.   // Get the errors using the global.getErrors function from conf.js
  38.   var errors = getErrors();
  39.   // errors was selected using $$. It returns a group (list), so you'll have to check pieces
  40.   it('should generate error messages', function() {
  41.     expect(errors).toBeDefined();
  42.     expect(errors.count()).toBe(1);
  43.   });
  44.   it('error messages should contain "sample error"', function() {
  45.     expect(errors.get(0).getText()).toContain('Sample error');
  46.   });
  47.   // Get the messages using the global.getMessages function from conf.js
  48.   var messages = getMessages();
  49.   it('should generate messages', function() {
  50.     expect(messages).toBeDefined();
  51.     expect(messages.count()).toBe(1);
  52.   });
  53.   it('messages should contain "sample message"', function() {
  54.     expect(messages.get(0).getText()).toContain('Sample message');
  55.   });
  56. });
  57.  
  58. // Special handling for unique elements such as 'Hold to activate' buttons
  59. describe('hold buttons', function(){
  60.   var holdButton = page.getHoldButton();
  61.   it('should give feedback on interaction', function(){
  62.     browser.actions().mouseDown(holdButton).perform();
  63.     browser.sleep(50);
  64.     expect(holdButton.getAttribute('class') ).toMatch('holding');
  65.     browser.actions().mouseUp(holdButton).perform();
  66.   });
  67.   it('should not complete in under .5 seconds', function(){
  68.     browser.actions().mouseDown(holdButton).perform();
  69.     browser.sleep(50);
  70.     browser.actions().mouseUp(holdButton).perform();
  71.     expect(holdButton.getAttribute('class') ).toMatch('hold-failure');
  72.   });
  73.   it('should complete eventually', function(){
  74.     browser.actions().mouseDown(holdButton).perform();
  75.     browser.sleep(2000);
  76.     browser.actions().mouseUp(holdButton).perform();
  77.     expect(holdButton.getAttribute('class') ).toMatch('hold-success');
  78.   });
  79. });

base.page.js

protractor/pages/base.page.js

The base page serves as a way to collect standard functionality that can later be inherited in other pages. These methods will be used by the style-guide.page.js file below.

  1. var BasePage = function(){};
  2.  
  3. BasePage.prototype.path = '#/';
  4.  
  5. BasePage.prototype.getPath = function(){
  6.     return path;
  7. };
  8.  
  9. // This is used in page to page reloads. You'll see more details on usage later.
  10. BasePage.prototype.getPathRegex = function(){
  11.     var regex = escapeRegExp(this.getPath());
  12.     return new RegExp(regex);
  13. };
  14.  
  15. /**
  16.  * Gets the page directly by loading a url. This will not reload the current page
  17.  */
  18. BasePage.prototype.get = function() {
  19.     var url = browser.baseUrl + this.getPath();
  20.  
  21.     if( browser.params.currentUrl !== url ) {
  22.         browser.params.currentUrl = url;
  23.         return browser.get(url);
  24.     } else {
  25.         return true;
  26.     }
  27. };
  28.  
  29. module.exports = BasePage;

style-guide.page.js

protractor/pages/style-guide.page.js

The previous spec file referenced page level functions frequently. The logic to find these elements and page variables are contained in page.js files to keep logic simple to maintain. See the protractor page object reference for more on the concept of why you need this.
This a basic spec definition file. It includes one or more 'definition' and 'it' blocks. These represent logic groupings of functional tests.

  1. 'use strict';
  2.  
  3. var BasePage = requirePage('base');
  4.  
  5. var StyleGuidePage = function(){};
  6.  
  7. StyleGuidePage.prototype = new BasePage();
  8.  
  9. StyleGuidePage.prototype.path = 'style-guide';
  10.  
  11. StyleGuidePage.prototype.getLoginLink = function(){
  12.     return element(by.id('header-login'));
  13. };
  14.  
  15. StyleGuidePage.prototype.getHoldButton = function(){
  16.     return $('button[hold-button]');
  17. };
  18.  
  19. module.exports = StyleGuidePage;

Additional Tests and Pages

Just to show some additional functionality to adapt from, and how I've handled page to page testing with the least duplication possible, here's the 'create new email that redirects to editing email' test. Note that I'm heavily reliant here on Protractor's natural handling. Any action that kicks off an angular action such as $http, causes testing to stop until Angular resolves.

new.spec.js

protractor/specs/email/new.page.js

  1. var NewEmailPage = requirePage('emails/new');
  2.  
  3. // Note that page.get HAS to be used inside an it() call.
  4. var page = new NewEmailPage();
  5.  
  6. describe('new email page', function() {
  7.   var titleInput, firstNameInput, templates, templateThumbnails, continueButton;
  8.  
  9.   it('should contain a required title input', function () {
  10.     page.get();
  11.     titleInput = page.getTitleInput();
  12.     expect(titleInput.isPresent()).toBeTruthy();
  13.     expect(titleInput.getAttribute('required')).toBeTruthy();
  14.   });
  15.  
  16.   it('should contain a required from name input', function () {
  17.     firstNameInput = page.getFromNameInput();
  18.     expect(firstNameInput.isPresent()).toBeTruthy();
  19.     expect(firstNameInput.getAttribute('required')).toBeTruthy();
  20.   });
  21.  
  22.   it('should default your from name based on your account', function(){
  23.     expect(firstNameInput.getAttribute('value')).toBeTruthy();
  24.   });
  25.  
  26.   it('should contain a list of templates available to you', function () {
  27.     templates = page.getTemplates();
  28.     expect(templates.count()).toBeGreaterThan(0);
  29.   });
  30.  
  31.   it('should set the templateId model when a thumbnail is clicked', function () {
  32.     templateThumbnails = page.getTemplateThumbnails();
  33.     templateThumbnails.get(0).click();
  34.     expect(templates.get(0).getAttribute('checked')).toBeTruthy();
  35.   });
  36.  
  37.   it('should enable the next button only after filling in title, from, and template', function () {
  38.     continueButton = page.getContinueButton();
  39.     expect(continueButton.getAttribute('disabled')).toBeTruthy();
  40.     titleInput.sendKeys( browser.params.testEmail.title );
  41.     templateThumbnails.get(0).click();
  42.     expect(continueButton.getAttribute('disabled')).not.toBeTruthy();
  43.   });
  44.  
  45.   it('should redirect you to the edit step when you click continue', function () {
  46.     var nextPage = page.getNextPage();
  47.     continueButton.click().then(function(){
  48.       var urlPromise = browser.getCurrentUrl();
  49.       expect( urlPromise ).toMatch( nextPage.getPathRegex());
  50.       urlPromise.then(function(url){
  51.           // Make sure to alert the BasePage we've moved
  52.           browser.params.currentUrl = url;
  53.       });
  54.     });
  55.   });
  56. });

new.page.js

protractor/pages/email/new.page.js

  1. 'use strict';
  2.  
  3. var BasePage = requirePage('base');
  4. var EditEmailPage = requirePage('emails/edit');
  5.  
  6. var NewEmailPage = function(){};
  7.  
  8. NewEmailPage.prototype = new BasePage();
  9.  
  10. NewEmailPage.prototype.path = 'emails/new';
  11.  
  12. NewEmailPage.prototype.getTitleInput = function(){
  13.     return element(by.model('title'));
  14. };
  15.  
  16. NewEmailPage.prototype.getFromNameInput = function(){
  17.     return element(by.model('fromName'));
  18. };
  19.  
  20. NewEmailPage.prototype.getTemplates = function(){
  21.     return $$('ul.templates li input[type=radio]');
  22. };
  23.  
  24. NewEmailPage.prototype.getTemplateThumbnails = function(){
  25.     return $$('ul.templates li label img');
  26. };
  27.  
  28. NewEmailPage.prototype.getContinueButton = function(){
  29.     return element(by.id('continue'));
  30. }
  31.  
  32. EditEmailPage.prototype.getNextPage = function(){
  33.   return new EditEmailPage();
  34. };
  35.  
  36. module.exports = NewEmailPage;

edit.page.js

protractor/pages/email/edit.page.js

  1. 'use strict';
  2.  
  3. var BasePage = requirePage('Base');
  4.  
  5. var EditEmailPage = function(){};
  6.  
  7. EditEmailPage.prototype = new BasePage();
  8.  
  9. EditEmailPage.prototype.path = 'emails/#';
  10.  
  11. EditEmailPage.prototype.getTitleInput = function(){
  12.     return element(by.model('title'));
  13. };
  14.  
  15. EditEmailPage.prototype.getFromNameInput = function(){
  16.     return element(by.model('fromName'));
  17. };
  18.  
  19. EditEmailPage.prototype.getContinueButton = function(){
  20.     return element(by.id('continue'));
  21. };
  22.  
  23. module.exports = EditEmailPage;

There are still a load of things I'm learning, mostly about DOM selection, and the limitations of Jasmine's expect() options. But it's getting clearer, and I'm really looking forward to being able to ensure the entire application works as expected in under 30 seconds.

One last little bonus, as it took me a while figure out it out:
Here's how to find a row with a link with a class of 'title' and text of 'Protractor Test Email', then return the parent row, and eventually the delete button:

  1. // The draft list. A nice simple id
  2. var drafts = element( by.id('drafts-list') );
  3. // The row, with the class and title match, and back up to the row
  4. var email = drafts.element(by.cssContainingText('.title', title)).element(by.xpath('ancestor::tr'));
  5. // And the child delete button
  6. var button = email.$('button.delete');

Now there's a good intro to how you can find your element...

Some gotchas

Having spent the last two weeks deep in testing, I thought I'd share findings with you:

  1. Always call browser.driver.manage().window().setSize() and provide random X and Y to simulate different devices during your conf.js onPrepare.
  2. You can and should test memory allocation when testing in chrome. Use browser.driver.executeScript('return window.performance.memory.usedJSHeapSize;').then(function(heapSize){ });
  3. browser.driver.executeScript('something()', element) will report "Failed: Maximum call stack size exceeded" if the argument you're passing in is too large. Always pass in using element.getWebElement() instead.
  4. browser.driver.executeScript('something()', element) will report "Failed: Maximum call stack size exceeded" if the return value is too large. Return only what you really need.
Tags: 

Comments

Hi,
How can I test inner app pages with token authorization? It always throw me on login page with test failure. I added in config file 'accessToken' with token value, but without success.

Hi Alexander,

Getting to pages you've restricted means going through the system you've setup to login. In the case of this application I've stubbed out, I'm handed my token from the backend people at the end of an OAuth exchange. Your login might be username or password based, and all you would have to do is walk through that process instead. I'd be happy to suggest a solution if you can tell me how your login system works, and where you store the persistent session token.

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.