The Unit Test provides integration with unit testing frameworks that support JUnit XML output. The only input for this type of test is a command to be run, which must produce a JUnit XML file at the location stored in the $CG_JUNIT_XML_LOCATION
environment variable. Students will then get a clear overview of all the tests that have been run, which ones failed, and the output that was produced in the case of failure.
The resulting score of this step is the fraction of successful tests divided by the total number of tests that were run, multiplied by the weight of the test.
All built in scripts will automatically produce the correct files in the right locations.
Unit Tests are useful if you want to run many tests but condense their output into a single entry in the test category. They are also useful when you already have a unit testing setup, and do not want to translate them all into individual AutoTest tests. Naturally, unit tests are a great way to assess individual functions and methods of larger programming assignments.
We provide wrapper scripts for a growing list of frameworks that handle everything for you, these are installed by default on AutoTest. If your unit testing framework is not in the list below, it is possible to manually create a small wrapper script for this to output the results in the correct location.
Want to know if your preferred testing framework is supported, or does a script not work as you need it to? Please send an email to [email protected] so we can discuss the possibilities!
Framework | Language(s) | Script name |
Check | C/C++ |
|
xUnit | C# |
|
Tasty (Quickcheck) | Haskell |
|
Junit 4 | Java |
|
Junit 5 | Java |
|
Jest | Javascript |
|
Mocha | Javascript |
|
MOxUnit | Matlab/Octave |
|
Nbgrader | Jupyter Notebook |
|
PHPUnit | PHP |
|
Pytest | Python 2+3 |
|
All wrapper scripts share a similar interface and procedure for running tests, which we describe below, replace SCRIPT
for the unit test script you want to use from the list above.
Installing: install the script in the global setup script to run by calling cg-SCRIPT compile
.
Compiling: For compiled languages it can be tricky to get the compiler to find all libraries and other dependencies needed to compile the students’ code and your tests. For these languages the wrapper scripts provide a compile
command that configures the compiler to be able to find the required libraries. The compile
command takes file names as arguments and compiles those files. Run the command cg-SCRIPT compile <file1> <file2> ...
to compile the student code and test classes.
Running: The wrapper scripts run the unit tests with their run
command.
The scripts extract the output location from $CG_JUNIT_XML_LOCATION
and then unset it before running the tests. The testing framework is then configured to output its results at that location and the tests are executed. Run the command cg-SCRIPT run <file1> <file2> ...
to compile the student code and test classes.
The exact arguments to the run
command of each script may differ between scripts. E.g. cg-junit5
is run via cg-junit5 run -- -c TestClass
.