🌐Web Development with Selenium

Overview

This guide illustrates how to automatically test web development assignments in CodeGrade using Jest and Selenium. Selenium is a powerful open-source testing tool for web development that allows you to functionally test the webpages created by your students. Below we are going to setup AutoTest V2 to test the student's implementation of the Simon game.

The student submission is assumed to contain an html file named index.html, while the behavior of the webpage is implemented via the game.js file. You can find out how to testcss properties here.

Example webpage submission rendered in CodeGrade

Setup

CodeGrade AutoTest V2 runs on Ubuntu (20.04 LTS) machines which you can flexibly configure. Common software is pre-installed, but you can also manually install software.

In the setup section of your AutoTest, you can install software or packages and upload any files you might need for testing. The setup section will build an image and is only run once for all submissions.

Step 1

First we need to install Node.js : drag the "Install Node" block to the setup step and select the preferred version in the dropdown.

Step 2

The second step is to install the additional software we need using a Script Block:

# Update ubuntu packages
sudo apt update

# Install Google Chrome
wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chrome-linux64.zip &> /dev/null
unzip chrome-linux64.zip &> /dev/null
rm -f ./chrome-linux64.zip
chmod +x ./chrome-linux64/chrome
sudo ln -s ${PWD}/chrome-linux64/chrome /usr/local/bin/chrome

# Install Chrome dependencies
sudo apt install libxkbcommon0 libgbm-dev -y

# Install Chromedriver
wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chromedriver-linux64.zip &> /dev/null
unzip chromedriver-linux64.zip &> /dev/null
rm -f chromedriver-linux64.zip
chmod +x ./chromedriver-linux64/chromedriver
sudo ln -s ${PWD}/chromedriver-linux64/chromedriver /usr/local/bin/chromedriver

# Install a headless display server
sudo apt install xvfb -y

# Install Jest 
npm install -g jest

This script installs the following software:

• From line 4 to 19 we download and install the Google Chrome web browser and its driver;

• In line 22 we install xvfb that makes it possible to run a web browser using a virtual display;

• In line 25 we install jest, the javascript unit test framework on which we run our selenium tests.

Step 3

Next we need to upload our fixtures, which in this case are:

• The package.json file that specifies all the node dependencies of the node project. This is where you have to list all the node packages the student may need for the project and you as a teacher may need to implement your tests. In our example we just need to install the testing dependencies:

  1. selenium-webdriver : the node package to run selenium in node.js;

  2. chromedriver: the node package to run the Google Chrome browser;

  3. jest-junit : the node package that enables jest to write the unit test results in a xml format.

• The game.test.js file that contains the jest unit tests that use selenium to functionally test the student webpages.

285B

package.json

2KB

game.test.js.zip

archive

The setup of thegame.test.js looks like this:

// import the selenium and chromedriver packages
const {Builder, By, Key, until} = require('selenium-webdriver');
require('chromedriver')

// This is the directory where the student's submission ends up.
// It can easily be changed to any URL.
const baseURL = "file://" + __dirname + "/";


// Change this to select another file to test.
const file = 'index.html'

const defaultTimeout = 1000;

// Four simple functions to ease DOM navigation
const getElementByName = async (driver, name, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.name(name)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

const getElementById = async (driver, id, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.id(id)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

const getElementByTag = async (driver, tag, timeout = defaultTimeout) => {
    const element = await driver.wait(until.elementLocated(By.tag(tag)), timeout);
    return await driver.wait(until.elementIsVisible(element), timeout);
};

function sleep(time) {
  return new Promise(resolve => setTimeout(() => resolve(), time));
}

// This is run before any test.
beforeAll(async () => {
    driver = await new Builder().forBrowser('chrome').build();
    await driver.get(baseURL + file);
});

// This makes sure the browser session exits.
afterAll(async () => {
    await driver.quit();
});

• In line 2 and 3 the selenium-webdriver and chromedriver packages are imported;

• In line 7 the constant baseURL points to the directory where the testing file is found;

• In line 11file is the name of the student's html file that we will open when starting our headless web browser. Here we assume that the student's file and the game.test.js file were moved in the same directory. Change this accordingly;

• From line 16 to line 29 the functions getElementbyName, getElementById and getElementByTag are defined. These functions help us navigate the webpages programatically;

• In line 31 a function sleep is defined. It simply awaits a given number of seconds;

• From line 36 to line 44 the beforeAll and afterAll functions are defined. These are special functions that jest uses for each test setup and teardown. In the setup we create a new webdriver object that points to the student initial webpage and in the teardown we make sure to properly exit the browser session.

Let's examine an actual unit test suite defined in the game.test.js file.

describe('Check setup.', () => {

        test('Check for title.', async () => {
            const title = await getElementById(driver, 'level-title');
            var title_attr = await title.getProperty('innerHTML');
            expect(title_attr).toEqual("Press A Key to Start");
        });

        test('Check for buttons.', async () => {
            const greenButton = await getElementById(driver, 'green');
            const blueButton = await getElementById(driver, 'blue');
            const yellowButton = await getElementById(driver, 'yellow');
            const redButton = await getElementById(driver, 'red');
            expect(greenButton).not.toBeUndefined();
            expect(redButton).not.toBeUndefined();
            expect(blueButton).not.toBeUndefined();
            expect(yellowButton).not.toBeUndefined();
        });

});

In line 3 we define the test Check for title. . This test checks that the HTML page created by the student contains a specific title element. Specifically:

• In line 4 we search for an HTML element whose id is level-titleand save it into a constant named title;

• In line 5 we access the innerHTML property that title is expected to have;

• In line 6 we check that the string used for the title is as expected

In line 9 we define the test Check for buttons. . This test checks that the HTML page contains elements corresponding to given ids. For example:

• In line 10 retrieves searches for an HTML element with the id green and saves it into a constant named greenButton;

• ln line 14 we check that the greenButton variable is actually defined.

Step 4

Assuming that all the students are going to use the same set of node packages indicated in the previously uploaded package.json file, it is convenient to install the node packages during the setup. This saves time for the students as the same installation won't have to be executed each time a new submission is run in the test phase. To do so simply drag and drop another Script Block and type:

# Move to the directory where the package.json file is
cd $UPLOADED_FILES

# Install the node packages indicated in the package.json file
npm install

The npm install creates a folder named node_modules inside the fixture folder. Later we can move this folder into the student directory where we run the tests.

AutoTest V2 Setup configuration for a Web Development assignment

Tests

Step 1

Before running the tests we need to move all the necessary fixtures into the student directory. This should be done at the top of the test phase using a Script Block that runs the following command:

mv $UPLOADED_FILES/node_modules $UPLOADED_FILES/package.json  $UPLOADED_FILES/game.test.js .

Step 2

We can finally run the selenium tests. To do so drag and drop a Custom Test Block where you can type the following commands:

# Run the unit tests and create a report file
xvfb-run jest --reporters=jest-junit  --testTimeout=1000

# Parse the test report file to display results and compute the grade
cg unit-tests junitxml junit.xml

• In line 2 we run jest, the javascript unit testing framework, prepending it with xvfb-run. This is needed as our selenium tests require a display to open the web browser; xvfb-run allows to use a virtual display. The option --reporters=jest-junit makes such that jest creates an xml report file containing the results of the tests. The option --testTimeout=10000 indicates a 10 second timeout for each tests.

Before and after running each test, jest performs setup and teardown operations that may throw a time out error with a message like:

Exceeded timeout of 1000 ms for a hook

To fix these errors you can increase the testTimeout option when invoking jest.

• In line 5 we use the CodeGrade builtin command cg to parse the xml report file produced by jest. This creates a visual display of the test results and computes a grade if required.

Optionally you can:

• connect the Custom Test Block to a rubric using a Connect Rubric Block;

• hide information to the students about the run Block, such as its configuration and output.

Custom Test Block implementing Selenium tests with Jest. The Block is connected to a rubric and both the Block's configuration and output are hidden to the students.