Unit Test

Search…
📋
Unit Test
Unit Tests work out of the box in AutoTest using the unit test step, simply upload your testing scripts and add it to your category.
The Unit Test provides integration with unit testing frameworks that support JUnit XML output. The only input for this type of test is a command to be run, which must produce a JUnit XML file at the location stored in the $CG_JUNIT_XML_LOCATION environment variable. Students will then get a clear overview of all the tests that have been run, which ones failed, and the output that was produced in the case of failure.
The resulting score of this step is the fraction of successful tests divided by the total number of tests that were run, multiplied by the weight of the test.
All built in scripts will automatically produce the correct files in the right locations.
Use cases
Unit Tests are useful if you want to run many tests but condense their output into a single entry in the test category. They are also useful when you already have a unit testing setup, and do not want to translate them all into individual AutoTest tests. Naturally, unit tests are a great way to assess individual functions and methods of larger programming assignments.
Compatible scripts
We provide wrapper scripts for a growing list of frameworks that handle everything for you, these are installed by default on AutoTest. If your unit testing framework is not in the list below, it is possible to manually create a small wrapper script for this to output the results in the correct location.
📋
Writing Unit Test wrapper scripts
Want to know if your preferred testing framework is supported, or does a script not work as you need it to? Please send an email to [email protected] so we can discuss the possibilities!
Supported frameworks
Framework
Language(s)
Script name
Check
C/C++
cg-check
xUnit
C#
cg-xunit
Tasty (Quickcheck)
Haskell
cg-quickcheck
Junit 4
Java
cg-junit4
Junit 5
Java
cg-junit5
Jest
Javascript
cg-jest
Mocha
Javascript
cg-mocha
MOxUnit
Matlab/Octave
cg-moxunit
Nbgrader
Jupyter Notebook
cg-nbgrader
PHPUnit
PHP
cg-phpunit
Pytest
Python 2+3
cg-pytest
Wrapper script usage
All wrapper scripts share a similar interface and procedure for running tests, which we describe below, replace SCRIPT for the unit test script you want to use from the list above.
Installing: install the script in the global setup script to run by calling cg-SCRIPT compile.
Compiling: For compiled languages it can be tricky to get the compiler to find all libraries and other dependencies needed to compile the students’ code and your tests. For these languages the wrapper scripts provide a compile command that configures the compiler to be able to find the required libraries. The compile command takes file names as arguments and compiles those files. Run the command cg-SCRIPT compile <file1> <file2> ...to compile the student code and test classes.
Running: The wrapper scripts run the unit tests with their run command.
The scripts extract the output location from $CG_JUNIT_XML_LOCATION and then unset it before running the tests. The testing framework is then configured to output its results at that location and the tests are executed. Run the command cg-SCRIPT run <file1> <file2> ...to compile the student code and test classes.
The exact arguments to the run command of each script may differ between scripts. E.g. cg-junit5 is run via cg-junit5 run -- -c TestClass.
​
Capture Points
Code Quality
Last modified 1mo ago
Copy link
Contents
Use cases
Compatible scripts