🤖Add Automatic Grading

Let CodeGrade AutoTest do the dirty work for you and automate (a part of) your grading!

Finally, we are adding automatic grading to our assignment. To create an AutoTest, click the Create button, or import a configuration from another assignment:

Look for a previous AutoTest configuration to import on the AutoTest tab in the Assignment Manager

Next, click on any of the languages below to get a language-specific guide for setting up autograding.

🐍pagePython☕pageJava📘pageJupyter Notebook📊pageR©️pageC➕pageC++🖥️pageC#

Is your language not listed? No worries, it will still be possible to automatically grade this. Contact us at [e-mail] for a specialized onboarding or follow the general guide below.

General guide

Step 1: Setup

CodeGrade AutoTest runs on Ubuntu 18.04.2 LTS machines which you can configure in any way that you want and have already pre installed most common software, like Python, Java and Git and much more.

In the setup section of your AutoTest, you can upload any files you might need for testing, these are called Fixtures and will be placed in the $FIXTURES directory on the Virtual Server.

The file structure of the server is like this:

$FIXTURES/
    All of your uploaded fixtures will be here.
$STUDENT/
    This is where the submission of student is placed. 
    Tests are executed from this directory.

After uploading any files you might need, you can run some setup to install any packages you might need.

• Global setup script: this is where you install your packages (for instance using sudo apt install PACKAGE_NAME). The global setup only runs once and is then cached, so the student submission won't be available here yet.

• Per-student setup script: this will run once when running the autograding for a student and will run before all tests. This is a good place to move $FIXTURES to the $STUDENT directory.

Use any command in the Global Setup Script field to install software or run your setup script

Step 2: Create your tests

Now that you have created the setup, it's time to create the actual tests. Do this by pressing the "Add Level" button and then the "Add Category" button.

All tests in AutoTest fill in a specific rubric category that you select. After selecting the category, you can start creating tests that will fill it in. Multiple test types are available in CodeGrade, which can be used together depending on your needs and wishes.

• Input/Output Tests: Specify the program you want to run (usually the student's program) and specify input and expected output.

• Run Program Tests: Run a command and if the command runs successfully this test will pass. Great for custom tests and compile commands.

• Unit Tests: Upload a unit test file as a $FIXTURE and automatically run unit tests on the code. We have several built-in options, follow one of the automatic grading guides above to find out more!

• Code Quality Tests: Automatic run static code analysis on the student's code. We have several built-in options, follow one of the automatic grading guides above to find out more!

• Capture Points Tests: Use your own custom grading script, upload it as a $FIXTURE, execute it in this test and make sure it outputs a float between 0.0 and 1.0, this score will then be the number of points a student receives.

• Checkpoints: Only execute tests below the checkpoint if the tests above the checkpoint reach a certain score. Great for separating simple and advanced tests.

Step 3: Start the AutoTest

Once you have created a configuration, press start and your AutoTest will start running. In the General Settings, you should have already uploaded a Test Submission, so you will see the results of this straight away.

Once it's started and your assignment is set to Open or Done, students can hand in and get immediate feedback!