​⚙ Configure AutoTest

CodeGrade AutoTest provides a server only for your tests, which you can configure in any way that you want! These servers run Ubuntu 18.04.2 LTS and have already pre installed most common software, like Python, Java and Git, of which you can find a list here. So for most basic assignments, configuring is not required!

​💾 If you want to built upon a previous AutoTest configuration, you can import it.

​🏗 Want to use an exotic language or package for a more advanced assignment? You can install that!

​📤 Do your students need specific input files to run their code? You can upload anything you want your AutoTest as a fixture!

​⚡ Turn on instant feedback for your students!

​🤖 Creating the tests

All tests in AutoTest fill in a specific rubric category that you select. After selecting the category, you can start creating tests that will fill it in. Multiple test types are available in CodeGrade, which can be used together depending on your needs and wishes. Find the tests available in CodeGrade below:

​✅ Want to check if a (student) program executes without errors, for instance to see if compilation works correctly or if the return code is succes, use a Run Program Test.

​⌨ Check whether your students' code returns the correct answer, given a certain input, use Input and Output Tests.

​🧩 Use your own custom grading scripts to review and check your students' code by using Capture Points tests to capture the output of those scripts.

​📋 Upload your unit tests and use the Unit Test step to run them automatically and display the output of all individual tests in a clear way to your students. We have built in support for many frameworks!​

​💎 Want to check the style and structure of your students' code automatically? Code Quality tests use linters to check this and display found errors directly in the code of the students.

​❎ Use Checkpoints to add conditions within your AutoTest categories, for instance, to only continue with subsequent tests if code compiles successfully or to add advanced tests that have to be unlocked.

​📈 For most assignments, just one AutoTest level is sufficient. But if your assignment is really elaborate or if you want to add advanced tests, you can create multiple levels with conditions between each other.

​💬 Your AutoTest test steps will have a default description that explains what command the test executes. You can replace this description with a custom one to make it more understandable for your students!

​👀 Limiting visibility to students

We believe the more details of AutoTest we display to students, the more feedback they get and the more engaged they are. By default, all tests are visible to students and students could programatically try to access fixtures. Learn how to limit visibility of tests and fixtures below.

​🧬 Advanced stuff!

We designed AutoTest to work with any workflow. Perhaps the following two tools are just what you need:

​✏ Generate output using AutoTest to then easily display and or manually assess. For instance if you want to generate GUI screenshots to assess, generate a testing report or run a Jupyter Notebook automatically.

​🔎 Find information about the submission you are grading in AutoTest in the submission metadata, useful to automatically deduct late days or seed random input data.

​🌐 Enable network access for your AutoTest categories, so that you can download requirements and your students can interact with the internet.

​💙 Need more help?

Setting up your first AutoTest can be a bit overwhelming, there are a lot of new tools and possibilities and you are changing the way you have been grading for years. We have distilled a document with some most common best practices in AutoTest for you:

Need more help? Or would you just like to call to confirm your AutoTest configuration makes sense? We are here for you!