➕Advanced C++ autograding

Discover the advanced autograding options available for C++ assignments

In this guide, we explore the advanced grading options available for C++ assignments. For more information about setting up a C++ assignment from scratch, see

➕Create your first C++ assignment

Google Test

Google Test is an industry-standard unit testing framework for C++. It is particularly useful for grading assignments that require students to write functions and classes. Using C++ unit tests offers several advantages over conventional IO tests, including the ability to use assertions, parametrize test cases, and provide better feedback for students. Consider the example submission below, which consists of a simple C++ class and its associated header file:

Calculator.cpp

Calculator.cpp

#include <gtest/gtest.h>
#include <stdexcept>

double add(double a, double b) 
{ 
    return a + b; 
}

double subtract(double a, double b)
{ 
    return a - b; 
}

double multiply(double a, double b)
{ 
    return a * b; 
}

double divide(double a, double b) 
{
    if (b == 0) throw std::runtime_error("Division by zero error");
    return a / b;
}

Calculator.hpp

In the following test file, we can define and create robust tests using Google Test assertions:

TestCalculator.cpp

TestCalculator.cpp

#include <gtest/gtest.h>
#include <stdexcept>
#include "Calculator.hpp"

// Test for Addition
TEST(CalculatorTests, AddTest) {
    RecordProperty("cg_name", "Test Addition of Two Numbers");
    EXPECT_DOUBLE_EQ(add(5, 3), 8) << "Addition should correctly add two numbers";
}

// Test for Subtraction
TEST(CalculatorTests, SubtractTest) {
    RecordProperty("cg_name", "Test Subtraction of Two Numbers");
    EXPECT_DOUBLE_EQ(subtract(5, 3), 2) << "Subtraction should correctly subtract second number from first";
}

// Test for Multiplication
TEST(CalculatorTests, MultiplyTest) {
    RecordProperty("cg_name", "Test Multiplication of Two Numbers");
    EXPECT_DOUBLE_EQ(multiply(5, 3), 15) << "Multiplication should correctly multiply two numbers";
}

// Test for Division
TEST(CalculatorTests, DivideTest) {
    RecordProperty("cg_name", "Test Division of Two Numbers");
    EXPECT_DOUBLE_EQ(divide(10, 2), 5) << "Division should correctly divide first number by second";
}

// Test for Division by Zero
TEST(CalculatorTests, DivideByZeroTest) {
    RecordProperty("cg_name", "Test Division by Zero Throws Error");
    EXPECT_THROW(divide(10, 0), std::runtime_error) << "Division by zero should throw a runtime_error";
}

int main(int argc, char **argv) {
    testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
}

Notice the use of:

• CodeGrade's builtin annotator RecordProperty that enables us to give custom names to our tests;

• custom error messages that are displayed in case of failure of any EXPECT assertion.

AutoTest Setup

As explained in our previous guide, we first need to install a C++ compiler. Our second step is to use a Script Block to install Google Test through the following Bash script:

sudo apt install cmake
git clone https://github.com/google/googletest.git -b v1.14.0
cd googletest        # Main directory of the cloned repository.
mkdir build          # Create a directory to hold the build output.
cd build
cmake ..             # Generate native build scripts for GoogleTest.
make
sudo make install    # Install in /usr/local/ by default

AutoTest Tests

In the Tests phase, we first need to use an Upload Files block to upload the following files:

• the testing file TestCalculator.cpp defined above;

• the CMake file CMakeLists.txt used to compile our tests:

CMakeLists.txt

cmake_minimum_required(VERSION 2.6)
project(Test)
IF(POLICY CMP0012)
    CMAKE_POLICY(SET CMP0012 NEW)
ENDIF()
# Locate GTest
find_package(GTest REQUIRED)
find_package(Threads REQUIRED)
include_directories(${GTEST_INCLUDE_DIRS})

# Link runTests with what we want to test and the GTest and pthread library
add_executable(runTests TestCalculator Calculator)
target_link_libraries(runTests ${GTEST_LIBRARIES} pthread)

In line 12, beside runTests we list the cpp files that need to be compiled together in order to run our tests, namely the testing file TestCalculator.cpp and our submission file Calculator.cpp .

When the student's file includes a main() function

If the submitted file defines its own main(), you'll get a duplicate symbol linking error. To fix this, you can wrap the student's main at link time using the GNU linker."

Add to TestCalculator.cpp:

extern "C" int __wrap_main(int x, char *y[]) {
    testing::InitGoogleTest(&x, y);
    return RUN_ALL_TESTS();
}

Add to CMakeLists.txt:

# Wrap main() from student code to avoid linking conflicts with GoogleTest
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--wrap=main")

Tests Compilation

We are now ready to compile our tests using a Script Block with the following script:

set -e
cp -r $UPLOADED_FILES/* . 
cmake -D CMAKE_CXX_COMPILER=clang++ CMakeLists.txt>/dev/null
make >/dev/null

Tests Execution

To finally run our tests we can use a Custom Test Block with the following script:

./runTests --gtest_output=json>/dev/null
res=$?
if [[ $res == 0 ]] || [[ $res == 1 ]]; then
    cg gtest parse test_detail.json
else 
    echo "Your tests did not execute correctly."
    exit $res
fi

An AutoTest configuration to run Google Test.

You can use a Connect Rubric Block to connect the tests to a rubric category.

You can hide the configuration of the blocks from the student.

To enhance the AutoTest clarity for students, it can be useful to run the Tests conditionally on the success of the compilation block.

Clang-Tidy

Clang-Tidy is an industry-standard linter for C++ that allows you to either enforce rules from various standard coding styles or to create your own set of rules. It is a useful tool for enforcing code styling best practices for beginner programmers.

AutoTest Setup

In the AutoTest Setup phase, in addition to typically installing a C++ compiler, we need to install Clang-Tidy with a simple Script Block that runs the following command:

sudo apt-get update && sudo apt-get install clang-tidy

AutoTest Tests

Running Clang-Tidy is done through the use of a Custom Test Block that runs the following script:

clang-tidy Calculator.cpp -checks=* 2>/dev/null | cg comments \
    '^(?P<file>/[^:]*):(?P<line>\d+):(?P<column>\d+): (?P<severity>[^:]*): (?P<message>.*) (\[(?P<code>[^\]]*)\])?$' \
     --origin clang-tidy \
     --ignore-regex '^([^/])' \
     --severities 'note:info,remark:info' \
     --deduction-warning '2' \
     --ignore-parser-errors 

This script first runs Clang-Tidy enforcing all the available checks through the option -checks=*, and then processes its output through the cg comments command. This will make such that all the coding style violations will be reported inline within the student's files in the submission folder.

You can select which checks to include/exclude among all the available ones. For this, we refer to the Clang Tidy documentation.

You can use a Connect Rubric Block to connect the tests to a rubric category. For a graded test you can decide the penalties to apply for each kind of violation (error, warning, etc...); for more information read here about the cg comments command.

You can hide the configuration of the blocks from the student.

Coding Style violations are reported inline within the student's submission file.

Conclusion

These advanced C++ testing options open the door to more rigorous testing methods and better student feedback. However, these are just the most commonly used methods. There are many more testing tools available that you can use with some simple setup. For more information, contact us at [e-mail].