IO Test

With an Input Output Test step, you can check whether the student code returns the right answer given a certain input.

The IO Test step is the most common test to create. This is a test in which you specify an input to a student program and an expected output. If the students' program returns the same output as expected, the test passes and they achieve points.

A (student) program can be specified to be run in the IO tests, then, you can specify all input and output combinations you want to check for this program. One IO Test step can have multiple pairs of input and output.

Each IO Test input and output pair can have its own weight, set this up under the Advanced Options tab.

Input can be given either as Input arguments, i.e. the arguments that are appended behind the Program to test, or as Input, which is simply standard input as if it would be written into the student program.

If you require files as input to your tests, upload these files as fixtures to find them in $FIXTURES/[FILE], where you can then use them as input arguments to your test.

Multiple options allow you to more flexibly compare the output of the program:

  • Case insensitive: ignore case in comparing the expected output and actual output.

  • Ignore trailing whitespace: trim all trailing whitespacing from the start and end.

  • Ignore all whitespace: all whitespace is ignored when comparing the student output with the expected output (including newlines).

  • Substring: the expected output should be a substring of the actual output.

  • Regex: the expected output is a Python3 Regex which should match with the actual output.

The options Ignore all whitespace and regex cannot be activated together.

The option substring is required when using the regex option.

Use cases

As the name suggests, IO Tests are used to check the output of a program given an input. IO Tests are very useful to check the functionality of simple and advanced programs. From checking for Hello, World! to checking if an advanced algorithm produces the right results. Additionally, IO Tests can be used to check the correct handling of incorrect input.