Links

Java

Create your first automatically graded Java assignment using CodeGrade's AutoTest v2
In this tutorial we will guide you through all the steps to create your very first Java assignment in CodeGrade. For this tutorial we will be considering the example below. Please make sure to save this file somewhere accessible on your computer:
Fibonacci.java
694B
Binary

Create an assignment

Firstly, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started:

Configure submission settings

The first step of setting up an assignment is to make sure that students can submit their code in the correct way. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option.
  1. 1.
    Navigate to the Assignment management menu by using the
    ⚙️
    icon at the right-hand side of the page header
  2. 2.
    Find the Submission settings under the General tab
  3. 3.
    Enable the File uploader and Editor submission methods by clicking the respective check boxes.
  4. 4.
    Enabling the Editor will reveal the Template files option. Download the Fibonacci.java file below and the upload it as a template file.
  5. 5.
    Finalize your settings by clicking the "Submit" button.
It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name.
For more information about submission methods, see Submission Settings.

Create a rubric

While you can always grade assignments directly by manually setting the Final grade, It's only possible to award points for your automatic tests with Rubrics. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students.
CodeGrade offers two types of rubric category:
  • Discrete category - Specify points in discrete steps (eg. 0, 5 or 10 points)
  • Continuous category - Specify points as a continuous scale (eg. 0 - 10)
  1. 1.
    Navigate to the Rubric tab in the Assignment management menu.
  2. 2.
    Select Create new rubric.
  3. 3.
    Create a new Continuous Category.
  4. 4.
    Provide the name "Compile test" as the Category name.
  5. 5.
    Under Category description write: "Full points are awarded if your program compiles without errors. If your program fails to compile, carefully check the comments placed on your code and try to resolve the errors."
  6. 6.
    Set the Min points to 0 and the Max points to 10.
  7. 7.
    Repeat this process three more times for the following rubric categories:
Name
Description
Points
Compile test
Check that your code compiles without errors.
0 - 10
Input/Output tests
Check that your program produces the correct output depending on the provided input.
0 - 50
Code structure tests
Check that you have used the expected code structures
0 - 20
Checkstyle code quality tests
Check that your program conforms to the Sun Checkstyle style guide.
0 - 20

Create automatic tests

Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work.
For this tutorial we will be setting up several automatic tests: Input/Output tests, Semgrep code structure tests, and Checkstyle code quality tests.

Setup

Before actually creating the tests, we need to make sure that the correct version of Java is installed.
  1. 1.
    Navigate to the Setup tab under the AutoTest settings.
  2. 2.
    Add an Install Java block to your configuration and select the latest version of Java from the drop-down menu.

Compile test

As with any compiled language like Java, each student's program must be compiled before we can run any other tests on it. We will also grade this test using the "Compile test" rubric category.
In this example we will go one step further than just showing the compiler errors. Instead we will parse the compiler output and if errors occur, we will place comments on the students code at the line that the error occurred on. This is done using the cg java compile command.
  1. 1.
    Navigate to the Tests tab under the AutoTest settings.
  2. 2.
    Add a Connect rubric block to your AutoTest configuration and select the "Compile test" rubric category from the drop-down menu.
  3. 3.
    Add a Script block to your AutoTest configuration and nest it within the Connect rubric block.
  4. 4.
    In the editor provided, add the following bash commands:
cg java compile Fibonacci.java

Input/Output tests

Input/Output tests (or IO tests for short) are a great way of checking a program's functionality by providing various input cases that should result in a different outputs. You can create IO test cases in AutoTest v2 using the IO Test wrapper block and its corresponding Full match, Substring match, and Regex match blocks. For more information about these blocks, see Create an IO Test.
  1. 1.
    Add a Connect Rubric block to your AutoTest configuration and select the "IO tests" rubric.
  2. 2.
    Add an IO test block to your AutoTest configuration and nest it in the Connect Rubric block. Run the compiled java file by writing the command java Fibonacci in the provided editor.
  3. 3.
    Add five Substring match blocks to your AutoTest configuration and nest them inside of the IO test block.
  4. 4.
    Copy and paste the info from the table below into the Input and Expected output fields in their respective match block:
Input
Expected output
1
1
3
1 1 2
5
1 1 2 3 5
0
invalid
-1
invalid

Code structure tests

Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax for reaching their solution. AutoTest v2 offers the Code structure test block and it's corresponding Positive match and Negative match blocks for running semgrep tests. For more information about these test blocks, see Create a Code Structure Test.
For this example we want to check that the students have defined the required functions in their code and that each function returns something. Here is the Semgrep rule we will be using to check the add() function:
1
rules:
2
- id: untitled_rule
3
pattern: |
4
for (...) {
5
...
6
}
7
message: Semgrep found a match
8
languages: [java]
9
severity: WARNING
  1. 1.
    Add a Connect rubric block to your test configuration and select the "Code structure tests" rubric category.
  2. 2.
    Add a Code structure test block to your test configuration and nest it within the Connect rubric block.
  3. 3.
    In the Student file input field, write "Fibonacci.java".
  4. 4.
    Add a Positive match block to your test configuration and nest it within the Code structure test block
  5. 5.
    Copy and paste the semgrep rule in the code block above into the provided editor. Make sure to delete/replace the base code provided in the editor.

Checkstyle code quality tests

Checkstyle is a code quality assessment tool (also known as a linter) for Java that automatically generates comments on a source file based on it's adherence to a style guide. You can easily create a checkstyle in AutoTest v2 using the Checkstyle block. The Checkstyle block deducts a percentage of marks from the test depending on the number and severity of style rules violated according to the selected style guide. It offers three style guide options, Sun, Google, and checkstyle's default ruleset, which are the three most common style guides applied to Java programs.
  1. 1.
    Add a Connect rubric block to your test configuration and select the "Checkstyle code quality tests" rubric category
  2. 2.
    Add a Checkstyle block to your test configuration and nest it within the Connect rubric block.
  3. 3.
    In the Student file input field write "Fibonacci.java" and select "Sun" from the Config template drop-down menu option.

Test and publish your AutoTests

It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with Snapshots. When you're ready to check your tests, simply press the Build snapshot button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission.
The first time you build a snapshot, CodeGrade will prompt you to upload a Test submission. Use the "Click here to upload files" option or drag-and-drop the Fibonacci.java file that you downloaded at the beginning of this tutorial and click "Submit".
Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the Publish snapshot button.

Conclusion

You have just built your first Automatically graded assignment, ready for your students to begin submitting work!
This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see Learn more or reach out to our support team at [email protected].