🌐Web Development with Selenium

Overview

This guide illustrates how to automatically test web development assignments in CodeGrade using Jest and Selenium.

Selenium is a powerful open-source testing tool for web development that allows you to test the webpages created by your students functionally. For instance, you may use Selenium to test that webpage page created by the student:

• Contains the required html elements such as tables, headings, and paragraphs;

• Responds to user interaction as expected, for example, by using buttons and inputs.

To automatically test the CSS properties of the student's page, see the dedicated guide here.

Below, we are going to set up AutoTest to test the student's implementation of the Simon game. The student submission includes:

• a html file named index.html;

• a javascript file namedgame.js that implements the webpage functionalities;

• the required jquery library in its minified version;

• a folder sounds that contains the mp3 files to play when buttons are pressed;

• optionally, a css file that defines css rules to enhance the webpage's appearance. This is not a necessary file here, as we won't be grading css rules in this guide.

Below, you can download the submission folder for this assignment:

82KB

submission.zip

archive

Example webpage submission rendered in CodeGrade

html files can be fully rendered within CodeGrade. This means that the students can see how their web pages are displayed and react right within their submission folder.

AutoTest Setup

In the setup section of your AutoTest, you can install software or packages and upload any files you might need for testing. The setup section will build an image and is only run once for all submissions.

Step 1

First, we need to install Node.js : drag the "Install Node" block to the setup step and select the preferred version in the dropdown.

Step 2

The second step is to install the additional software we need using a Script Block:

# Update Ubuntu packages
sudo apt update

# Install Google Chrome
wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chrome-linux64.zip &> /dev/null
unzip chrome-linux64.zip &> /dev/null
rm -f ./chrome-linux64.zip
chmod +x ./chrome-linux64/chrome
sudo ln -s ${PWD}/chrome-linux64/chrome /usr/local/bin/chrome

# Install Chrome dependencies
sudo apt install libxkbcommon0 libgbm-dev -y

# Install Chromedriver
wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chromedriver-linux64.zip &> /dev/null
unzip chromedriver-linux64.zip &> /dev/null
rm -f chromedriver-linux64.zip
chmod +x ./chromedriver-linux64/chromedriver
sudo ln -s ${PWD}/chromedriver-linux64/chromedriver /usr/local/bin/chromedriver

# Install a headless display server
sudo apt install xvfb -y

# Install Jest 
npm install -g jest eslint eslint-config-standard eslint-plugin-promise eslint-plugin-import eslint-plugin-n eslint-detailed-reporter

This script installs the following software:

• the Google Chrome web browser and its driver;

• xvfb , a display server that makes it possible to run a web browser using a virtual display;

• jest, the javascript unit test framework on which we run our selenium tests.

Step 3

Next, we need to upload our fixtures, which in this case are:

• The package.json file that specifies all the node dependencies of the node project. This is where you have to list all the node packages the student may need for the project and you as a teacher may need to implement your tests. In our example, we just need to install the testing dependencies:

  1. selenium-webdriver : the node package to run selenium in node.js;

  2. chromedriver: the node package to run the Google Chrome browser;

  3. jest-junit : the node package that enables jest to write the unit test results in a xml format.

{
  "name": "game",
  "version": "1.0.0",
  "description": "test for Simon game",
  "main": "game.js",
  "scripts": {
    "test": "jest"
  },
  "author": "codegrade",
  "license": "ISC",
  "dependencies": {
    "selenium-webdriver":"",
    "chromedriver":"",
    "jest-junit":""
  }
}

• The game.test.js file that contains the jest unit tests that use selenium to functionally test the student webpage.

game.test.js

Below you have the testing file game.test.js:

const {Builder, By, Key, until} = require('selenium-webdriver');
require('chromedriver')

// This is the directory where the student's submission ends up.
// It can easily be changed to any URL.
const baseURL = "file://" + __dirname + "/";


// Change this to select another file to test.
const file = 'index.html'

const defaultTimeout = 1000;

// Four simple functions to ease DOM navigation
const getElementByName = async (driver, name, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.name(name)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

const getElementById = async (driver, id, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.id(id)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

const getElementByTag = async (driver, tag, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.tagName(tag)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

function sleep(time) {
  return new Promise(resolve => setTimeout(() => resolve(), time));
}

// This is run before any test.
beforeAll(async () => {
    driver = await new Builder().forBrowser('chrome').build();
    await driver.get(baseURL + file);
});

// This makes sure the browser session exits.
afterAll(async () => {
    await driver.quit();
});


describe('Check setup.', () => {

        test('Check for title.', async () => {
            const title = await getElementById(driver, 'level-title');
            var title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual("Press A Key to Start");
        });

        test('Check for buttons.', async () => {
            const greenButton = await getElementById(driver, 'green');
            const blueButton = await getElementById(driver, 'blue');
            const yellowButton = await getElementById(driver, 'yellow');
            const redButton = await getElementById(driver, 'red');
            expect(greenButton).not.toBeUndefined();
            expect(redButton).not.toBeUndefined();
            expect(blueButton).not.toBeUndefined();
            expect(yellowButton).not.toBeUndefined();
        });

});

describe('Check initialization.', () => {

        test('Check for game start.', async () => {
            const body = await driver.findElement(By.css('body'));
            await body.sendKeys('RETURN');
            await sleep(2000);
            let title = await getElementById(driver, 'level-title');
            let title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual('Level 1');
        });

});

describe('Check Game Mechanics.', () => {

        test('Check correct press.', async () => {
            const gamePattern = await driver.executeScript('return gamePattern;');
            const buttonOne = await getElementById(driver, gamePattern[0]);
            await buttonOne.click();
            await sleep(2000);
            let title = await getElementById(driver, 'level-title');
            let title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual('Level 2');
        });

        test('Check another correct press.', async () => {
            const gamePattern = await driver.executeScript('return gamePattern;');
            const buttonOne = await getElementById(driver, gamePattern[0]);
            const buttonTwo = await getElementById(driver, gamePattern[1]);
            await buttonOne.click();
            await buttonTwo.click();
            await sleep(2000);
            let title = await getElementById(driver, 'level-title');
            let title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual('Level 3');
        });

        test('Check wrong press.', async () => {
            const gamePattern = await driver.executeScript('return gamePattern;');
            let buttonOne = null;
            if (gamePattern[0] != 'green') {
              buttonOne = await getElementById(driver, 'green');
            } else {
              buttonOne = await getElementById(driver, 'red');
            }
            await buttonOne.click();
            await sleep(2000);
            var title = await getElementById(driver, 'level-title');
            var title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual('Game Over, Press Any Key to Restart');
        });

}, defaultTimeout);

The game.test.js file contains a first setup part and a second part where we define the actual tests.

Setup

• In lines 2 and 3 the selenium-webdriver and chromedriver packages are imported;

• In line 7 the constant baseURL points to the directory where the testing file is found;

• In line 11file is the name of the student's html file that we will open when starting our headless web browser. Here, we assume that the student's file and the game.test.js file was moved to the student directory so that testing and the tested files are in the same directory.

• From line 16 to line 29 the functions getElementbyName, getElementById and getElementByTag are defined. These functions help us navigate the web pages programmatically;

• In line 31 a function sleep is defined. It simply awaits a given number of seconds;

• From line 36 to line 44 the beforeAll and afterAll functions are defined. These are special functions that jest uses for each test setup and teardown. In the setup, we create a new web driver object that points to the student's initial webpage, and in the teardown, we make sure to properly exit the browser session.

Tests

Let's now examine some actual tests.

In line 49 we define the test Check for title. . This test checks that the html page created by the student contains a specific title element. Specifically:

• In line 50 we search for an HTML element whose id is level-titleand save it into a constant named title;

• In line 51 we access the innerHTML property that title is expected to have;

• In line 52 we check that the string used for the title is as expected.

In line 55 we define the test Check for buttons. . This test checks that the page contains elements corresponding to specific html ids. For example:

• line 56 searches for an html element with the id green and saves it into a constant named greenButton;

• ln line 60 we check that the greenButton variable is properly defined.

Step 4

Assuming that all the students are going to use the same set of node packages indicated in the previously uploaded package.json file, it is convenient to install the node packages during the AutoTest Setup. This saves time for the students as the same installation won't have to be executed each time a new submission is run. For this, simply drag and drop another Script Block and type:

# Move to the directory where the package.json file is
cd $UPLOADED_FILES

# Install the node packages indicated in the package.json file
npm install

The npm install creates a folder named node_modules inside the fixture folder. Later we can move this folder into the student directory where we run the tests.

AutoTest V2 Setup configuration for a Web Development assignment

AutoTest Tests

Step 1

Before running the tests we need to move all the necessary fixtures into the student directory. This should be done at the top of the Tests phase using a Script Block that runs the following command:

mv $UPLOADED_FILES/node_modules $UPLOADED_FILES/package.json  $UPLOADED_FILES/game.test.js .

Step 2

We can finally run the selenium tests. To do so , drag and drop a Custom Test Block and run the following commands:

# Run the unit tests and create a report file
xvfb-run jest --reporters=jest-junit  --testTimeout=1000

# Parse the test report file to display results and compute the grade
cg unit-tests junitxml junit.xml

• In line 2 we run jest, the javascript unit testing framework, prepending it with xvfb-run. This is needed as our selenium tests require a display to open the web browser; xvfb-run allows to use a virtual display. The option --reporters=jest-junit makes such that jest creates an xml report file containing the results of the tests. The option --testTimeout=10000 indicates a 10 second timeout for each tests.

• In line 5 we use the CodeGrade builtin command cg to parse the xml report file produced by jest. This visually displays the test results and computes a grade if required.

Optionally, you can:

• connect the Custom Test Block to a rubric using a Connect Rubric Block;

• hide information to the students about the run Block, such as its configuration and output.

Custom Test Block implementing Selenium tests with Jest. The Block is connected to a rubric and both the Block's configuration and output are hidden to the students.