The resulting score of this test is determined by the amount of comments that were produced by the linter. Each comment has a severity level, which is determined by the linter, and for each different severity level you can configure an amount of points to be subtracted from the score.
There are four severity levels: fatal, error, warning, and info.
The final score is the score left over from subtracting the penalty of each comment from the initial score of 100, interpreted as a percentage, multiplied by the weight of the step.
A simple Java Code Quality (Checkstyle) test with a configuration file uploaded as a fixture
Press the "AddCode Quality" button to add a new Code Quality test to your Test Category.
Select the linter you want to run from the dropdown menu, or select "custom" if you want to run a linter that is not present in the list.
If you selected a linter from the list, you can enter a configuration file to be used by the linter, and any extra arguments to be passed to the linter program. If you chose "custom", enter the program to be ran.
Configure the penalties for each severity level, as described above.
Want to check for only specific style or structure violation? Most linters allow you to specify what to test on! E.g. in flake8 you can use the --select E501 as argument to only check for line length.
Some linters such as Checkstyle require a configuration file.