Creating Code Quality Tests

Use linters in CodeGrade to automatically assess style, structure and overall code quality using the Code Quality step.

The Code Quality test runs a static code analysis tool or other program that produces feedback on the code, which is stored and displayed inline in the student's code.

Want to use a code analysis tool that is not (yet) built in? Follow this guide to create your own wrapper script to use it in our Code Quality step!

The resulting score of this test is determined by the amount of comments that were produced by the linter. Each comment has a severity level, which is determined by the linter, and for each different severity level you can configure an amount of points to be subtracted from the score.

There are four severity levels: fatal, error, warning, and info.

The final score is the score left over from subtracting the penalty of each comment from the initial score of 100, interpreted as a percentage, multiplied by the weight of the step.

A simple Java Code Quality (Checkstyle) test with a configuration file uploaded as a fixture

1. Press the "Add Code Quality" button to add a new Code Quality test to your Test Category.

2. Select the linter you want to run from the dropdown menu, or select "custom" if you want to run a linter that is not present in the list.

3. If you selected a linter from the list, you can enter a configuration file to be used by the linter, and any extra arguments to be passed to the linter program. If you chose "custom", enter the program to be ran.

4. Configure the penalties for each severity level, as described above.

Want to check for only specific style or structure violation? Most linters allow you to specify what to test on! E.g. in flake8 you can use the --select E501 as argument to only check for line length.

Some linters such as Checkstyle require a configuration file.