🌐Web Development with Selenium
Overview
This guide illustrates how to automatically test web development assignments in CodeGrade using Jest and Selenium. Selenium is a powerful open-source testing tool for web development that allows you to functionally test the webpages created by your students. Below we are going to setup AutoTest V2 to test the student's implementation of the Simon game.
The student submission is assumed to contain an html file named index.html
, while the behavior of the webpage is implemented via the game.js
file.
You can find out how to testcss
properties here.
Setup
CodeGrade AutoTest V2 runs on Ubuntu (20.04 LTS) machines which you can flexibly configure. Common software is pre-installed, but you can also manually install software.
In the setup section of your AutoTest, you can install software or packages and upload any files you might need for testing. The setup section will build an image and is only run once for all submissions.
Step 1
First we need to install Node.js : drag the "Install Node" block to the setup step and select the preferred version in the dropdown.
Step 2
The second step is to install the additional software we need using a Script Block:
This script installs the following software:
From line 4 to 19 we download and install the Google Chrome web browser and its driver;
In line 22 we install
xvfb
that makes it possible to run a web browser using a virtual display;In line 25 we install
jest
, the javascript unit test framework on which we run our selenium tests.
Step 3
Next we need to upload our fixtures, which in this case are:
The
package.json
file that specifies all the node dependencies of the node project. This is where you have to list all the node packages the student may need for the project and you as a teacher may need to implement your tests. In our example we just need to install the testing dependencies:selenium-webdriver
: the node package to run selenium in node.js;chromedriver
: the node package to run the Google Chrome browser;jest-junit
: the node package that enables jest to write the unit test results in axml
format.
The
game.test.js
file that contains the jest unit tests that use selenium to functionally test the student webpages.
The setup of thegame.test.js
looks like this:
In line 2 and 3 the
selenium-webdriver
andchromedriver
packages are imported;In line 7 the constant
baseURL
points to the directory where the testing file is found;In line 11
file
is the name of the student's html file that we will open when starting our headless web browser. Here we assume that the student's file and thegame.test.js
file were moved in the same directory. Change this accordingly;From line 16 to line 29 the functions
getElementbyName
,getElementById
andgetElementByTag
are defined. These functions help us navigate the webpages programatically;In line 31 a function
sleep
is defined. It simply awaits a given number of seconds;From line 36 to line 44 the
beforeAll
andafterAll
functions are defined. These are special functions that jest uses for each test setup and teardown. In the setup we create a new webdriver object that points to the student initial webpage and in the teardown we make sure to properly exit the browser session.
Let's examine an actual unit test suite defined in the game.test.js
file.
In line 3 we define the test Check for title.
. This test checks that the HTML page created by the student contains a specific title element. Specifically:
In line 4 we search for an HTML element whose id is
level-title
and save it into a constant namedtitle
;In line 5 we access the
innerHTML
property thattitle
is expected to have;In line 6 we check that the string used for the title is as expected
In line 9 we define the test Check for buttons.
. This test checks that the HTML page contains elements corresponding to given ids. For example:
In line 10 retrieves searches for an HTML element with the id
green
and saves it into a constant namedgreenButton
;ln line 14 we check that the
greenButton
variable is actually defined.
Step 4
Assuming that all the students are going to use the same set of node packages indicated in the previously uploaded package.json
file, it is convenient to install the node packages during the setup.
This saves time for the students as the same installation won't have to be executed each time a new submission is run in the test phase. To do so simply drag and drop another Script Block and type:
The npm install
creates a folder named node_modules
inside the fixture folder. Later we can move this folder into the student directory where we run the tests.
Tests
Step 1
Before running the tests we need to move all the necessary fixtures into the student directory. This should be done at the top of the test phase using a Script Block that runs the following command:
Step 2
We can finally run the selenium tests. To do so drag and drop a Custom Test Block where you can type the following commands:
In line 2 we run
jest
, the javascript unit testing framework, prepending it withxvfb-run
. This is needed as our selenium tests require a display to open the web browser;xvfb-run
allows to use a virtual display. The option--reporters=jest-junit
makes such thatjest
creates an xml report file containing the results of the tests. The option--testTimeout=10000
indicates a 10 second timeout for each tests.
Before and after running each test, jest
performs setup and teardown operations that may throw a time out error with a message like:
Exceeded timeout of 1000 ms for a hook
To fix these errors you can increase the testTimeout
option when invoking jest.
In line 5 we use the CodeGrade builtin command
cg
to parse the xml report file produced byjest
. This creates a visual display of the test results and computes a grade if required.
Optionally you can:
connect the Custom Test Block to a rubric using a Connect Rubric Block;
hide information to the students about the run Block, such as its configuration and output.
Last updated