This guide illustrates how to automatically test web development assignments in CodeGrade using Jest and Selenium.
is a powerful open-source testing tool for web development that allows you to test the webpages created by your students functionally. For instance, you may use Selenium to test that webpage page created by the student:
Contains the required html elements such as tables, headings, and paragraphs;
Responds to user interaction as expected, for example, by using buttons and inputs.
To automatically test the CSS properties of the student's page, see the dedicated guide .
Below, we are going to set up AutoTest to test the student's implementation of the Simon game. The student submission includes:
a html file named index.html;
a javascript file namedgame.js that implements the webpage functionalities;
the required jquery library in its minified version;
a folder sounds that contains the mp3 files to play when buttons are pressed;
optionally, a css file that defines css rules to enhance the webpage's appearance. This is not a necessary file here, as we won't be grading css rules in this guide.
Below, you can download the submission folder for this assignment:
html files can be fully rendered within CodeGrade. This means that the students can see how their web pages are displayed and react right within their submission folder.
AutoTest Setup
In the setup section of your AutoTest, you can install software or packages and upload any files you might need for testing. The setup section will build an image and is only run once for all submissions.
xvfb , a display server that makes it possible to run a web browser using a virtual display;
Step 3
The package.json file that specifies all the node dependencies of the node project. This is where you have to list all the node packages the student may need for the project and you as a teacher may need to implement your tests. In our example, we just need to install the testing dependencies:
selenium-webdriver : the node package to run selenium in node.js;
chromedriver: the node package to run the Google Chrome browser;
jest-junit : the node package that enables jest to write the unit test results in a xml format.
The game.test.js file that contains the jest unit tests that use selenium to functionally test the student webpage.
game.test.js
Below you have the testing file game.test.js:
const {Builder, By, Key, until} = require('selenium-webdriver');
require('chromedriver')
// This is the directory where the student's submission ends up.
// It can easily be changed to any URL.
const baseURL = "file://" + __dirname + "/";
// Change this to select another file to test.
const file = 'index.html'
const defaultTimeout = 1000;
// Four simple functions to ease DOM navigation
const getElementByName = async (driver, name, timeout = defaultTimeout) => {
const element = await driver.wait(until.elementLocated(By.name(name)), timeout);
return await driver.wait(until.elementIsVisible(element), timeout);
};
const getElementById = async (driver, id, timeout = defaultTimeout) => {
const element = await driver.wait(until.elementLocated(By.id(id)), timeout);
return await driver.wait(until.elementIsVisible(element), timeout);
};
const getElementByTag = async (driver, tag, timeout = defaultTimeout) => {
const element = await driver.wait(until.elementLocated(By.tagName(tag)), timeout);
return await driver.wait(until.elementIsVisible(element), timeout);
};
function sleep(time) {
return new Promise(resolve => setTimeout(() => resolve(), time));
}
// This is run before any test.
beforeAll(async () => {
driver = await new Builder().forBrowser('chrome').build();
await driver.get(baseURL + file);
});
// This makes sure the browser session exits.
afterAll(async () => {
await driver.quit();
});
describe('Check setup.', () => {
test('Check for title.', async () => {
const title = await getElementById(driver, 'level-title');
var title_attr = await title.getProperty('innerHTML');
expect(title_attr).toEqual("Press A Key to Start");
});
test('Check for buttons.', async () => {
const greenButton = await getElementById(driver, 'green');
const blueButton = await getElementById(driver, 'blue');
const yellowButton = await getElementById(driver, 'yellow');
const redButton = await getElementById(driver, 'red');
expect(greenButton).not.toBeUndefined();
expect(redButton).not.toBeUndefined();
expect(blueButton).not.toBeUndefined();
expect(yellowButton).not.toBeUndefined();
});
});
describe('Check initialization.', () => {
test('Check for game start.', async () => {
const body = await driver.findElement(By.css('body'));
await body.sendKeys('RETURN');
await sleep(2000);
let title = await getElementById(driver, 'level-title');
let title_attr = await title.getProperty('innerHTML');
expect(title_attr).toEqual('Level 1');
});
});
describe('Check Game Mechanics.', () => {
test('Check correct press.', async () => {
const gamePattern = await driver.executeScript('return gamePattern;');
const buttonOne = await getElementById(driver, gamePattern[0]);
await buttonOne.click();
await sleep(2000);
let title = await getElementById(driver, 'level-title');
let title_attr = await title.getProperty('innerHTML');
expect(title_attr).toEqual('Level 2');
});
test('Check another correct press.', async () => {
const gamePattern = await driver.executeScript('return gamePattern;');
const buttonOne = await getElementById(driver, gamePattern[0]);
const buttonTwo = await getElementById(driver, gamePattern[1]);
await buttonOne.click();
await buttonTwo.click();
await sleep(2000);
let title = await getElementById(driver, 'level-title');
let title_attr = await title.getProperty('innerHTML');
expect(title_attr).toEqual('Level 3');
});
test('Check wrong press.', async () => {
const gamePattern = await driver.executeScript('return gamePattern;');
let buttonOne = null;
if (gamePattern[0] != 'green') {
buttonOne = await getElementById(driver, 'green');
} else {
buttonOne = await getElementById(driver, 'red');
}
await buttonOne.click();
await sleep(2000);
var title = await getElementById(driver, 'level-title');
var title_attr = await title.getProperty('innerHTML');
expect(title_attr).toEqual('Game Over, Press Any Key to Restart');
});
}, defaultTimeout);
The game.test.js file contains a first setup part and a second part where we define the actual tests.
Setup
In lines 2 and 3 the selenium-webdriver and chromedriver packages are imported;
In line 7 the constant baseURL points to the directory where the testing file is found;
In line 11file is the name of the student's html file that we will open when starting our headless web browser. Here, we assume that the student's file and the game.test.js file was moved to the student directory so that testing and the tested files are in the same directory.
From line 16 to line 29 the functions getElementbyName, getElementById and getElementByTag are defined. These functions help us navigate the web pages programmatically;
In line 31 a function sleep is defined. It simply awaits a given number of seconds;
From line 36 to line 44 the beforeAll and afterAll functions are defined. These are special functions that jest uses for each test setup and teardown. In the setup, we create a new web driver object that points to the student's initial webpage, and in the teardown, we make sure to properly exit the browser session.
Tests
Let's now examine some actual tests.
In line 49 we define the test Check for title. . This test checks that the html page created by the student contains a specific title element. Specifically:
In line 50 we search for an HTML element whose id is level-titleand save it into a constant named title;
In line 51 we access the innerHTML property that title is expected to have;
In line 52 we check that the string used for the title is as expected.
In line 55 we define the test Check for buttons. . This test checks that the page contains elements corresponding to specific html ids. For example:
line 56 searches for an html element with the id green and saves it into a constant named greenButton;
ln line 60 we check that the greenButton variable is properly defined.
Step 4
Assuming that all the students are going to use the same set of node packages indicated in the previously uploaded package.json file, it is convenient to install the node packages during the AutoTest Setup.
This saves time for the students as the same installation won't have to be executed each time a new submission is run. For this, simply drag and drop another Script Block and type:
# Move to the directory where the package.json file is
cd $UPLOADED_FILES
# Install the node packages indicated in the package.json file
npm install
The npm install creates a folder named node_modules inside the fixture folder. Later we can move this folder into the student directory where we run the tests.
# Run the unit tests and create a report file
xvfb-run jest --reporters=jest-junit --testTimeout=1000
# Parse the test report file to display results and compute the grade
cg unit-tests junitxml junit.xml
In line 2 we run jest, the javascript unit testing framework, prepending it with xvfb-run. This is needed as our selenium tests require a display to open the web browser; xvfb-run allows to use a virtual display. The option --reporters=jest-junit makes such that jest creates an xml report file containing the results of the tests. The option --testTimeout=10000 indicates a 10 second timeout for each tests.
Optionally, you can:
First, we need to install Node.js : drag the to the setup step and select the preferred version in the dropdown.
The second step is to install the additional software we need using a :
, the javascript unit test framework on which we run our selenium tests.
Next, we need to , which in this case are:
Before running the tests we need to move all the necessary fixtures into the .
This should be done at the top of the Tests phase using a Script Block that runs the following command:
We can finally run the selenium tests. To do so , drag and drop a and run the following commands:
In line 5 we use the CodeGrade builtin command to parse the xml report file produced by jest. This visually displays the test results and computes a grade if required.
connect the Custom Test Block to a rubric using a ;
to the students about the run Block, such as its configuration and output.
AutoTest V2 Setup configuration for a Web Development assignment
Custom Test Block implementing Selenium tests with Jest. The Block is connected to a rubric and both the Block's configuration and output are hidden to the students.
