cg-junit5wrapper script to run unit tests on Java code with JUnit 5. We will write the script in Python because it's a bit more flexible and powerful than a shell script, and also because AutoTest runners have the
cg_at_utilslibrary installed automatically, which is a Python package that provides utility functions for writing wrapper scripts like these.
install: Installs any dependencies of your wrapper script that is not available by default on CodeGrade's AutoTest runners. This is usually executed in the global setup script, and may not be necessary at all if all dependencies are already installed by default (see full list here).
compile: Perform compilation if necessary. May be skipped if the programming language is not a compiled one, e.g. a scripting language like Python.
run: Run the tests and generate the test output. The output is a JUnit XML file (note: there does not exist a formal specification of the JUnit XML format, but the specification found in this link is supported by CodeGrade, except for the
<properties>tag). This file must be written to the randomized location specified in the
$CG_JUNIT_XML_LOCATIONenvironment variable (available in AutoTest, just like e.g. the
cg_at_utilspackage provides a
make_wrapperfunction that takes three functions — one for each of the stages described above — to help reduce the amount of boilerplate code you have to write. This uses the
typerlibrary to help defining the command line arguments to your script, which are automatically inferred from the types of the arguments of the functions you define.
.jarfile somewhere on the machine. We will also need a program to combine the multiple
.xmlfiles that are generated by JUnit 5 into one. We will use the
junitparserPython library for this.
apt, but the versions of JUnit 5 and the junitparser library on
aptare quite outdated, so we will fetch the JUnit 5
.jarof version 1.6.2 from the Maven repository and save it in
$FIXTURES/junit5.jarand we will use
pipto install junitparser. We will use utility functions provided by the
cg_at_utilspackage to simplify these steps!
compilestage is fairly straightforward. We must update the
CLASSPATHenvironment variable to include the student directory containing the code we want to compile, and the location to the
junit5.jarwe have downloaded in the
installstage, so that the Java compiler can find the required classes and link them correctly.
cg_at_utilslibrary provides a function for obtaining the student directory, so we will use that. It also has a utility function for modifying
PATH-like environment variables — that is, variables containing a list of paths to search, delimited by a
:. Because we will need the same class path in the
runstage we have extracted this to a separate function named
javac File1.java File2.java .... We use the
cg_at_utilsto run the command, and pass
check=Trueto make it quit with an error code if the compilation failed.
runstage is usually the most complicated one. Because there is no formal specification of the JUnit XML format, most unit test runners invent their own formats which all closely resemble each other, but are not quite the same most of the time. Because of this it can happen that you need to write your own converter for the output format of your test runner to a format that is recognized by CodeGrade. Luckily the XML produced by JUnit 5 is almost exactly what CodeGrade expects, so we just need to write a very simple converter function to remove the
<properties>tags that are not supported:
runfunction we set the class path using the function we defined in the previous stage. We will also have to know the location where we must write the JUnit XML file containing the test results to. We can get that from the
CG_JUNIT_XML_LOCATIONenvironment variable, but the
cg_at_utilslibrary also provides a function
junit_xml.get_locationto do this for us. We pass as arguments
delete=Trueto this function, which will also unset the
CG_JUNIT_XML_LOCATIONenvironment variable after it has read it. It is generally recommended to do this, so that students can not get its value from within their code.
java -jar /opt/my-junit5/junit5.jar ...and the arguments that were passed on the command line, e.g.
-c MyTestClass. We also explicitly pass the
-cpargument to JUnit 5 to ensure that it is consistent with the value we have set in the
CLASSPATHenvironment variable and we create a temporary directory where it can write the test reports to. Finally, we pass
run_cmdswill exit after running all the commands it was given, even if none of them produced an error.
jupitermode which can run JUnit 5 tests, and
vintagemode which can handle JUnit 4 tests. One report file is produced per mode, so we need to combine them into one file.
my-junit5, but you can name it however you want) and make it executable with
chmodto be able to run it in the AutoTest containers.
.javafiles in the current directory. Put it in the Per student setup script field, or create a Run Program step in an AutoTest category and put it in the Program to test field.
MyTest.javain the fixtures directory to the current directory before compiling, as it contains the tests we want to run, which must also be compiled.
runfunction, we specified that
my-junit5 runaccepts a list of string arguments, all of which we pass directly to our invocation of the JUnit 5 test runner. In the example below these are
--is ignored by the
typerlibrary, and tells it to stop parsing command line arguments, which is necessary here, because otherwise
typerwill try to parse the
-cargument and throw an error because we have not defined a
-cargument to our
weightattribute to the
<testcase>tags in the generated JUnit XML report. The weight must be a decimal number, and does not have to be present for all test cases. A weight of
1will be assumed for tests without a