🐍Advanced Python autograding

Discover the advanced autograding options available for Python assignments

In this guide, we explore the advanced grading options available for Python assignments. For more information about setting up a Python assignment from scratch, see:

🐍Create your first Python assignment

Simple Python Test

The simple Python test block is a great tool for building assignments geared towards beginner programmers. It allows you to insert your students' code into a Python script, enabling more grading and feedback options than just IO testing.

Example 1

One use of the Simple Python Test block is to ask students to write basic Python code without writing a complete program. Consider the example submission below:

reversed_sentence.py

'''
    Assume that a string variable called "sentence" has already been declared for you.
    Write a statement using string slicing that reverses the string and
    saves it to a variable called "reversed_sentence".
'''

reversed_sentence = sentence[::-1]

As you can see, this program would not be able to run on its own since sentence is not defined. Using the Simple Python Test, we can define the variable and insert the student's code to make a complete program. The student's code is inserted in the Simple Python Test block using the comment # CG_INSERT filename.

sentence = "This is another sentence"
answer = sentence[::-1]

# CG_INSERT reversed_sentence.py

print("your answer was: ", reversed_sentence)
print("correct answer is: ", answer)

if reversed_sentence == answer:
    "Your answer was correct!"
    exit(0)
else:
    "Your answer was incorrect!"
    exit(1)

Example 2

Another use case for the Simple Python Test block is to provide better feedback to students using the cg_feedback_helpers Python package. Consider the example submission below:

bubble_sort.py

'''
    Assume you have been provided with a list of integers called `numbers`.
    Sort the list from smallest to largest using the bubble sort method.
'''

for i in range(length - 1):
    for j in range((length - 1) - i):
        if numbers[j] > numbers[j + 1]:
            numbers[j], numbers[j + 1] = numbers[j + 1], numbers[j]

Using the cg_feedback_helpers module with the Simple Python Test block we can run our students' code in the context of a full program and use assertions to provide better feedback. The cg_feedback_helpers package can be installed in the AutoTest setup using the command python -m pip install cg_feedback_helpers.

from cg_feedback_helpers import asserter, NO_FEEDBACK

numbers = [34, 12, 99, 54, 28]
sorted_numbers = [12, 28, 34, 54, 99]

# CG_INSERT bubble_sort.py

print("your answer was: ", numbers)
print("correct answer is: ", sorted_numbers)

# Assert the answer is correct
asserter.equals(
    numbers,
    sorted_numbers,
    positive_feedback = "Your answer was correct!",
    negative_feedback = "Your answer was incorrect!"
)

# Display the feedback
asserter.emit_success(feedback=NO_FEEDBACK)

Pytest

Pytest is the industry standard unit testing framework for Python. The Pytest block is a dedicated test block for running Pytest unit tests on your students' submissions. This block is great for grading assignments in which students create their own functions. Consider the example submission below:

calculator.py

'''
    Write a function for the arithmetic operations add, subtract, multiply, and divide.
    Each function should have two arguments and should return the correct answer.
    The divide function should raise the ValueError exception if an attempt is made to divide by 0.
'''

def add(x, y):
    ans = x + y
    return ans


def subtract(x, y):
    ans = x + y
    return ans


def multiply(x, y):
    ans = x * y
    return ans

def divide(x, y):
    if y == 0:
        raise ValueError("Can not divide by zero!")
    ans = x / y
    return ans
    

Using the Pytest block, we can easily create multiple test cases to assess the functionality of the square() function. Additionally, we can use the cg_pytest_reporter PyPi module to enhance the feedback from our test cases. The @name and @description decorators allow us to make the test cases clearer and more readable for students. The @weight decorator allows us to adjust the grading weight of separate test cases.

import pytest
from calculator import add, subtract, multiply, divide
from cg_pytest_reporter import name, description, weight

@name("Test add()")
@description("Check that add() produces the correct result")
def test_add():
    assert add(2,3) == 5
    assert add(1,-1) == 0
    assert add(-1,-1) == -2

@name("Test subtract()")
@description("Check that subtract() produces the correct result")
def test_subtract():
    assert subtract(5, 2) == 3
    assert subtract(1, -1) == 2
    assert subtract(-1, -1) == 0

@name("Test multiply()")
@description("Check that multiply() produces the correct result")
def test_multiply():
    assert multiply(3, 2) == 6
    assert multiply(1, -1) == -1
    assert multiply(-1, -1) == 1

@name("Test divide()")
@description("Check that divide() produces the correct result or raises an error")
@weight(2)
def test_divide():
    assert divide(10, 2) == 5
    assert divide(1, -1) ==-1
    assert divide(-1, -1) == 1
    assert divide(5, 2) == 2.5

    with pytest.raises(ValueError):
        divide(10, 0)

The feedback from each test case is also beautifully rendered, making it easy for students to read.

Flake8

Flake8 is an industry-standard linter for Python that enforces the PEP8 style guide. It is a useful tool for enforcing code styling best practices for beginner programmers. The Flake8 block is an off-the-shelf testing tool that doesn't require any additional setup or configuration other than providing the name of the file to be tested. The Flake8 block generates comments directly on students' code. Each target line is highlighted according to the severity of the comment and the comment can be read by hovering over the line number with the mouse cursor. The Flake8 block also works in the built-in editor making it a powerful combination for beginners.

You can also update the configuration of Flake8 to ignore or select certain code style rules. This can be done by editing the config field. For example, this configuration would ignore the E203 error (Whitespace before ','.) as defined by the pycodestyle package:

[flake8]
extend-ignore = E203

Conclusion

These advanced Python testing options open the door to more rigorous testing methods and better student feedback. However, these are just the readily available code blocks. There are many more testing tools and packages available that you can use with some simple setup. For more information, contact us at [e-mail].