🖥️Advanced C# autograding

Discover the advanced autograding options available for C# assignments

In this guide, we explore the advanced grading options available for C# assignments. For more information about setting up a C# assignment from scratch, see:

🖥️Create your first C# assignment

Place Compile or Build errors as comments with

Using the .NET framework, we compile the students' files using the command, which produces an output that potentially contains compilation errors or warnings. We can report and highlight these inline within the student submission using the custom CodeGrade command cg comments .

For example, to build a dotnet console project, we can use a Script Block that runs the following commands:

cd ~/console_project

# copy the student's files into the project directory
cp $STUDENT/*.cs .

# save the output produced when building the project
OUTPUT=$(dotnet build --disable-build-servers /clp:NoSummary)

# use the cg comments command to produce inline comments
echo "$OUTPUT"  | cg comments generic \
    '^(?P<file>[^\(]*)\((?P<line>\d+),(?P<column>\d+)\):\s*(?P<severity>error|warning|info)\s+(?P<code>CS\d+):(?P<message>[^[]*).*$' \
    --severities 'Error:error,Warning:warning' \
    --origin "build" \
    --ignore-parser-errors

# exit the script according to the outcome of the build command
echo "$OUTPUT" | sed '/warning/d'
if [[ "$OUTPUT" == *"error"* ]]; then
    exit 1
fi

This will result in compilation errors or warnings being visible to the students within the student submission folder, as shown below:

Unit Tests with xUnit

xUnit is the standard unit testing tool within the .NET Framework and can be used to test .NET languages such as C#. xUnit is especially appropriate for grading assignments that require students to write functions and classes. xUnit unit tests offer several advantages over conventional IO tests including the ability to use assertions, parametrize test cases, and provide custom feedback for students. For an overview of the testing attributes available with xUnit, we recommend this reference.

Consider the following Calculator.cs submission that implements a simple arithmetic calculator:

using System;

public class Calculator
{
    public double Add(double a, double b)
    {
        return a + b;
    }

    public double Subtract(double a, double b)
    {
        return a - b;
    }

    public double Multiply(double a, double b)
    {
        return a * b;
    }

    public double Divide(double a, double b)
    {
        if (b == 0)
        {
            Console.WriteLine("Error: Division by zero is not allowed.");
            return double.NaN;
        }
        return a / b;
    }
}

Our tests are defined in the following CalculatorTests.cs file:

using Xunit;
using System;

namespace CalculatorTests
{

    public class CalculatorTests
    {
        private Calculator calculator = new Calculator();

        [Fact(DisplayName = "[5] Addition Test")]
        public void AdditionTest()
        {
            double result = calculator.Add(5, 3);
            Assert.Equal(8, result);
        }

        [Fact(DisplayName = "[3] Subtraction Test")]
        public void SubtractionTest()
        {
            double result = calculator.Subtract(5, 3);
            Assert.Equal(1, result);
        }

        [Fact(DisplayName = "[5] Multiplication Test")]
        public void MultiplicationTest()
        {
            double result = calculator.Multiply(5, 3);
            Assert.Equal(15, result);
        }

        [Fact(DisplayName = "[3] Division Test")]
        public void DivisionTest()
        {
            double result = calculator.Divide(6, 3);
            Assert.Equal(2, result);
        }

        [Fact(DisplayName = "Division by Zero Test")]
        public void DivisionByZeroTest()
        {
            Assert.Equal(double.NaN, calculator.Divide(6, 0));
        }
    }
}

As displayed above, we can use the xUnit's Fact attribute to set a custom name for each test. Moreover, by using naming the test as "[n] My Custom Test Name", we can also set a custom weight equal to n , with the default weight being 1.

Setup

In order to run unit tests with xUnit, in the Setup Phase we first have to install dotnet using the corresponding block and create an xUnit project.

After installing dotnet, use a Script Block and type the following bash script:

# create a folder with an xUnit project
mkdir xunit_project
cd xunit_project
dotnet new xunit

# remove the template testing class created by default
rm UnitTest1.cs

# install tool to create a test xml report in the junit format
dotnet add package XunitXml.TestLogger --version 3.1.20
dotnet tool install --global dotnet-xunit-to-junit --version 6.0.0

# make the installed tool available from the command line by adding it to the path
echo 'export PATH=$PATH:~/.dotnet/tools' >> ~/.cg_bash_env

If the class you want to test contains a main method you should add the following line at the bottom of the script:

# Modify the xUnit project configuration so that dotnet does not create an additional Main Method for the Unit Test class
sed  -i '/<PropertyGroup>/a <GenerateProgramFile>false</GenerateProgramFile>' xunit_project.csproj

Tests

Project Build

Once we are in the Tests phase, before running the actual tests, we have to:

• Upload the fixture file that defines the xUnit tests;

• Move both the xUnit testing file and the student submission files to the xUnit project folder;

• Compile the student's code by building the xUnit Project.

After using an Upload File block to upload the xUnit testing file, use a Script Block to run the following script:

# move to the xunit project folder
cd ~/xunit_project

# add both the submission and testing files to the xUnit project
cp $STUDENT/*.cs .
cp $FIXTURES/*.cs .

# build the project
OUTPUT=$(dotnet build --disable-build-servers /clp:NoSummary)

# display compilation comments inline
echo "$OUTPUT" | sed 's|/Files|/Student|g' | cg comments generic \
    '^(?P<file>[^\(]*)\((?P<line>\d+),(?P<column>\d+)\):\s*(?P<severity>error|warning|info)\s+(?P<code>CS\d+):(?P<message>[^[]*).*$' \
    --severities 'Error:error,Warning:warning' \
    --origin "build" \
    --ignore-parser-errors

# exit the script according to the outcome of the build command
echo "$OUTPUT" | sed '/warning/d'
if [[ "$OUTPUT" == *"error"* ]]; then
    exit 1
fi

Notice that:

• you can hide the script's content from the student using a Hide block as shown in the image above.

• we can report potential compilation errors inline into the student's submission, as explained at the top of this page.

Test execution

We are finally ready to run our tests. For this, we can use a Custom Test Block.

The Custom Test Block runs the following commands:

# move to the xUnit project folder
cd ~/xunit_project

# run the xUnit Tests and log the test results in junit xml format
dotnet test --disable-build-servers --logger:"xunit;LogFilePath=results.xml" 

# convert the xml in the junit format
dotnet xunit-to-junit "results.xml" "junitResults.xml"

# use the cg command to parse the test result and display the result to the student
cg junitxml junitResults.xml

# shut down the dotnet background process
dotnet build-server shutdown

As shown in the image above, you can optionally:

• hide the configuration of the Custom Test Block from the student;

• run the tests only if the Compilation step succeeded using a Run-If Block.

The results of the tests will be shown to the student as below: