Setup automatic grading

Let CodeGrade AutoTest do the dirty work for you and automate (a part of) your grading!

​⚙ Configure AutoTest
CodeGrade AutoTest will fill in rubric categories that you select. Make sure to set up a rubric before creating your AutoTest. Learn how to do that here.​
CodeGrade AutoTest provides a server only for your tests, which you can configure in any way that you want! These servers run Ubuntu 18.04.2 LTS and have already pre installed most common software, like Python, Java and Git, of which you can find a list here. So for most basic assignments, configuring is not required!
​💾 If you want to built upon a previous AutoTest configuration, you can import it.
Importing an old configuration
/for-teachers/creating-automatic-tests/importing-an-old-configuration
​🏗 Want to use an exotic language or package for a more advanced assignment? You can install that!
Installing new software
/for-teachers/creating-automatic-tests/installing-new-software
​📤 Do your students need specific input files to run their code? You can upload anything you want your AutoTest as a fixture!
Uploading files (or fixtures)
/for-teachers/creating-automatic-tests/uploading-files-or-fixtures
​⚡ Turn on instant feedback for your students!
Give students Instant Feedback
/for-teachers/creating-automatic-tests/advanced-autograding/giving-students-instant-feedback
​🤖 Creating the tests
All tests in AutoTest fill in a specific rubric category that you select. After selecting the category, you can start creating tests that will fill it in. Multiple test types are available in CodeGrade, which can be used together depending on your needs and wishes. Find the tests available in CodeGrade below:
​✅ Want to check if a (student) program executes without errors, for instance to see if compilation works correctly or if the return code is succes, use a Run Program Test.
Creating Run Program Tests
/for-teachers/creating-automatic-tests/creating-run-program-tests
​⌨ Check whether your students' code returns the correct answer, given a certain input, use Input and Output Tests.
IO Tests work great if you want to check the functionality of a whole program or part of a program. Use Unit Tests if you want to check on a per function or per method level.
Creating Input and Output Tests
/for-teachers/creating-automatic-tests/creating-input-and-output-tests
​🧩 Use your own custom grading scripts to review and check your students' code by using Capture Points tests to capture the output of those scripts.
A Capture Points Test works great for any custom grading script or custom wrapper script, using a linter or unit testing framework? Use our Code Quality or Unit Test steps instead.
Creating Capture Points Tests
/for-teachers/creating-automatic-tests/creating-capture-points-tests
​📋 Upload your unit tests and use the Unit Test step to run them automatically and display the output of all individual tests in a clear way to your students. We have built in support for many frameworks!​
Is your framework not supported out of the box? Creating a wrapper script for it is easy and we would love to help you out with that. Just contact us!​
Creating Unit Tests
/for-teachers/creating-automatic-tests/creating-unit-tests
​💎 Want to check the style and structure of your students' code automatically? Code Quality tests use linters to check this and display found errors directly in the code of the students.
Creating Code Quality Tests
/for-teachers/creating-automatic-tests/creating-code-quality-tests
​❎ Use Checkpoints to add conditions within your AutoTest categories, for instance, to only continue with subsequent tests if code compiles successfully or to add advanced tests that have to be unlocked.
Creating Checkpoints
/for-teachers/creating-automatic-tests/creating-checkpoints
​📈 For most assignments, just one AutoTest level is sufficient. But if your assignment is really elaborate or if you want to add advanced tests, you can create multiple levels with conditions between each other.
Creating multiple levels
/for-teachers/creating-automatic-tests/advanced-autograding/creating-multiple-levels
​💬 Your AutoTest test steps will have a default description that explains what command the test executes. You can replace this description with a custom one to make it more understandable for your students!
Adding custom descriptions for tests
/for-teachers/creating-automatic-tests/advanced-autograding/adding-custom-descriptions-for-tests
​👀 Limiting visibility to students
We believe the more details of AutoTest we display to students, the more feedback they get and the more engaged they are. By default, all tests are visible to students and students could programatically try to access fixtures. Learn how to limit visibility of tests and fixtures below.
Have performance heavy tests? 🏋 It is also useful to hide these, so that they are only executed after the deadline and thus do not stall continuous feedback progress for students.
Limiting visibility to students
/for-teachers/creating-automatic-tests/advanced-autograding/limiting-visibility-to-students
​⏯ Testing, starting and stopping your AutoTest
​🧪 Hooray! You have made your AutoTest configuration! It is now time to test it! But it is even better to iteratively test it while creating it.
Testing your AutoTest
/for-teachers/creating-automatic-tests/testing-your-autotest
​▶ If everything works as you expected, you can start your AutoTest and give your students the instant feedback they deserve!
Starting your AutoTest
/for-teachers/creating-automatic-tests/starting-your-autotest
​🔁 Need to make any changes while your AutoTest is turned on? Simply stop it, make your changes, and restart it without any worries!
Stopping or changing your AutoTest
/for-teachers/creating-automatic-tests/stopping-or-changing-your-autotest
​🧬 Advanced stuff!
We designed AutoTest to work with any workflow. Perhaps the following two tools are just what you need:
​✏ Generate output using AutoTest to then easily display and or manually assess. For instance if you want to generate GUI screenshots to assess, generate a testing report or run a Jupyter Notebook automatically.
Generating output automatically
/for-teachers/creating-automatic-tests/advanced-autograding/generating-output-automatically
​🔎 Find information about the submission you are grading in AutoTest in the submission metadata, useful to automatically deduct late days or seed random input data.
Accessing submission metadata
/for-teachers/creating-automatic-tests/advanced-autograding/accessing-submission-metadata
​🌐 Enable network access for your AutoTest categories, so that you can download requirements and your students can interact with the internet.
Enabling network access
/for-teachers/creating-automatic-tests/advanced-autograding/enabling-network-access
​💙 Need more help?
Setting up your first AutoTest can be a bit overwhelming, there are a lot of new tools and possibilities and you are changing the way you have been grading for years. We have distilled a document with some most common best practices in AutoTest for you:
Best Practices in AutoTest
/for-teachers/creating-automatic-tests/best-practices-in-autotest
Need more help? Or would you just like to call to confirm your AutoTest configuration makes sense? We are here for you!
✉️ Contact us

Using the correct state for your assignment

Importing an old configuration

Last updated 5 months ago

Setup automatic grading

​⚙ Configure AutoTest

​💾 If you want to built upon a previous AutoTest configuration, you can import it.

​🏗 Want to use an exotic language or package for a more advanced assignment? You can install that!

​📤 Do your students need specific input files to run their code? You can upload anything you want your AutoTest as a fixture!

​⚡ Turn on instant feedback for your students!

​🤖 Creating the tests

​✅ Want to check if a (student) program executes without errors, for instance to see if compilation works correctly or if the return code is succes, use a Run Program Test.

​⌨ Check whether your students' code returns the correct answer, given a certain input, use Input and Output Tests.

​🧩 Use your own custom grading scripts to review and check your students' code by using Capture Points tests to capture the output of those scripts.

​📋 Upload your unit tests and use the Unit Test step to run them automatically and display the output of all individual tests in a clear way to your students. We have built in support for many frameworks!​

​💎 Want to check the style and structure of your students' code automatically? Code Quality tests use linters to check this and display found errors directly in the code of the students.

​❎ Use Checkpoints to add conditions within your AutoTest categories, for instance, to only continue with subsequent tests if code compiles successfully or to add advanced tests that have to be unlocked.

​📈 For most assignments, just one AutoTest level is sufficient. But if your assignment is really elaborate or if you want to add advanced tests, you can create multiple levels with conditions between each other.

​💬 Your AutoTest test steps will have a default description that explains what command the test executes. You can replace this description with a custom one to make it more understandable for your students!

​👀 Limiting visibility to students

​⏯ Testing, starting and stopping your AutoTest

​🧪 Hooray! You have made your AutoTest configuration! It is now time to test it! But it is even better to iteratively test it while creating it.

​▶ If everything works as you expected, you can start your AutoTest and give your students the instant feedback they deserve!

​🔁 Need to make any changes while your AutoTest is turned on? Simply stop it, make your changes, and restart it without any worries!

​🧬 Advanced stuff!

​✏ Generate output using AutoTest to then easily display and or manually assess. For instance if you want to generate GUI screenshots to assess, generate a testing report or run a Jupyter Notebook automatically.

​🔎 Find information about the submission you are grading in AutoTest in the submission metadata, useful to automatically deduct late days or seed random input data.

​💙 Need more help?

⚙ Configure AutoTest

💾 If you want to built upon a previous AutoTest configuration, you can import it.

🏗 Want to use an exotic language or package for a more advanced assignment? You can install that!

📤 Do your students need specific input files to run their code? You can upload anything you want your AutoTest as a fixture!

⚡ Turn on instant feedback for your students!

🤖 Creating the tests

✅ Want to check if a (student) program executes without errors, for instance to see if compilation works correctly or if the return code is succes, use a Run Program Test.

⌨ Check whether your students' code returns the correct answer, given a certain input, use Input and Output Tests.

🧩 Use your own custom grading scripts to review and check your students' code by using Capture Points tests to capture the output of those scripts.

📋 Upload your unit tests and use the Unit Test step to run them automatically and display the output of all individual tests in a clear way to your students. We have built in support for many frameworks!

💎 Want to check the style and structure of your students' code automatically? Code Quality tests use linters to check this and display found errors directly in the code of the students.

❎ Use Checkpoints to add conditions within your AutoTest categories, for instance, to only continue with subsequent tests if code compiles successfully or to add advanced tests that have to be unlocked.

📈 For most assignments, just one AutoTest level is sufficient. But if your assignment is really elaborate or if you want to add advanced tests, you can create multiple levels with conditions between each other.

💬 Your AutoTest test steps will have a default description that explains what command the test executes. You can replace this description with a custom one to make it more understandable for your students!

👀 Limiting visibility to students

⏯ Testing, starting and stopping your AutoTest

🧪 Hooray! You have made your AutoTest configuration! It is now time to test it! But it is even better to iteratively test it while creating it.

▶ If everything works as you expected, you can start your AutoTest and give your students the instant feedback they deserve!

🔁 Need to make any changes while your AutoTest is turned on? Simply stop it, make your changes, and restart it without any worries!

🧬 Advanced stuff!

✏ Generate output using AutoTest to then easily display and or manually assess. For instance if you want to generate GUI screenshots to assess, generate a testing report or run a Jupyter Notebook automatically.

🔎 Find information about the submission you are grading in AutoTest in the submission metadata, useful to automatically deduct late days or seed random input data.

💙 Need more help?