# Grading test coverage with JaCoCo

Writing good unit tests is a skill students need to develop, which means you need a way to grade the tests they write — not just whether they pass, but how thoroughly they exercise the code. JaCoCo is a Java code coverage library that measures what percentage of a reference implementation is exercised by a set of unit tests. The higher the coverage, the more of the reference code the student's tests are touching.

This guide walks through uploading a reference implementation alongside the JaCoCo jars, running the student's tests under the JaCoCo agent, and converting the resulting coverage data into a numeric grade.

{% hint style="info" %}
This guide covers a scenario where **students write the tests** and you grade them against a reference implementation you provide. For the opposite scenario — instructor-written tests grading student code — see [JUnit5](/automatic-grading-guides/java/grading-with-junit5-and-checkstyle.md) guide.
{% endhint %}

### Before you start

You will need the following files ready to upload:

* `jacocoagent.jar` — the JaCoCo Java agent. Attached to the JVM with `-javaagent`, it instruments classes at runtime and records which lines are executed.
* `jacococli.jar` — the JaCoCo command-line tool. It converts the raw coverage data into an XML report that can be graded.
* `cov_grade.sh` — a shell script that reads the XML report and prints a numeric grade for the rubric. See the [Appendix](#appendix-cov_grade.sh) for the full script.
* Your reference implementation (e.g. `Calculator.java`).

Both JaCoCo jars can be downloaded from the [official JaCoCo release page](https://www.jacoco.org/jacoco/).

### Step 1: Install the JUnit console launcher

JaCoCo instruments the JVM directly, so the standard `cg junit5` helpers cannot be used here. Instead, download the JUnit5 platform console launcher jar from Maven Central during Setup.

1. Navigate to the **Setup** tab under the AutoTest settings and install JUnit5.

<figure><img src="/files/qBdmQmG8xyfkrGcOLFRo" alt="" width="375"><figcaption><p>AutoTest Setup to install JUnit5 using Script block</p></figcaption></figure>

2. Navigate to the **Test** tab and add an **Allow Internet** block.
3. Add a **Script** block inside the **Allow Internet** block and paste the following:

{% code overflow="wrap" %}

```bash
curl -O https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.10.0/junit-platform-console-standalone-1.10.0.jar
mv junit-platform-console-standalone-1.10.0.jar junit-platform-console-standalone.jar
```

{% endcode %}

<figure><img src="/files/jki0rs3TDBgZcKPgbR1P" alt="" width="563"><figcaption><p>An Allow Internet block wrapping a Script that downloads the JUnit5 console launcher.</p></figcaption></figure>

{% hint style="info" %}
The **Allow Internet** block is required in **Test** because `curl` needs to reach Maven Central. Renaming the jar drops the version number so the rest of your scripts can reference a stable path.
{% endhint %}

### Step 2: Compile the student's tests against a reference implementation

1. In **Tests** tab, add an **Upload Files** block and attach `jacocoagent.jar`, `jacococli.jar`, `cov_grade.sh`, and your reference implementation (e.g. `Calculator.java`).
2. Add a **Connect Rubric** block and select the appropriate rubric category from the drop-down menu.
3. Add a **Script** block nested inside the **Connect Rubric** block and paste the following:

```bash
mv $UPLOADED_FILES/* .
javac -cp ".:$STUDENT:junit-platform-console-standalone.jar" \
  Calculator.java \
  $STUDENT/CalculatorTest.java
```

<figure><img src="/files/hmsacEfLiY3Tzo7u8U54" alt="" width="563"><figcaption><p>A Connect Rubric containing the Compilation script.</p></figcaption></figure>

{% hint style="info" %}
Moving the uploaded files into the working directory first ensures that the compiler and the JaCoCo jars can all find each other on the classpath.
{% endhint %}

### Step 3: Run the tests under JaCoCo and grade the coverage

This step uses two **Custom Test** blocks nested inside the same **Connect Rubric** block from Step 2.

#### Custom Test 1 — Run student unit tests

Add a **Custom Test** block and paste the following:

```bash
# Run tests with JaCoCo agent, output to JUnit XML
java -javaagent:./jacocoagent.jar \
  -cp ".:$STUDENT:junit-platform-console-standalone.jar" \
  org.junit.platform.console.ConsoleLauncher \
  execute \
  --disable-banner \
  --details=none \
  --reports-dir=junitxml \
  --select-class=CalculatorTest

cg unit-tests junitxml junitxml/*.xml
```

{% hint style="info" %}
The `-javaagent` flag attaches the JaCoCo agent to the JVM, which instruments the reference implementation's classes at runtime and writes the raw coverage data to `jacoco.exec`. `cg unit-tests` parses the JUnit XML output so pass/fail information appears in the submission view.
{% endhint %}

#### Custom Test 2 — Grade code coverage

Add a second **Custom Test** block immediately after and paste the following:

```bash
# Directory layout required by JaCoCo's report command
mkdir src tests compiled
mv Calculator.java src
mv Calculator.class compiled
mv $STUDENT/CalculatorTest.class tests
rm $STUDENT/CalculatorTest.java

# Generate the XML coverage report
java -jar jacococli.jar report jacoco.exec \
  --classfiles compiled \
  --sourcefiles src \
  --xml cov.xml

# Grade based on the XML report
./cov_grade.sh cov.xml
```

<figure><img src="/files/8Jkv0dVQG5xSdAOe6ws0" alt="" width="563"><figcaption><p>The Custom Test blocks for running student unit tests and grading code coverage.</p></figcaption></figure>

{% hint style="warning" %}
The directory structure (`src`, `tests`, `compiled`) is required by JaCoCo's `report` command. Skipping or renaming these folders will cause the report step to fail.
{% endhint %}

### Conclusion

You have set up an automatically graded assignment that measures how thoroughly students test a reference implementation. For background on standard instructor-written JUnit 5 tests, or for adding Checkstyle to your Java assignments, see the guides below. For questions about custom grading scripts or coverage thresholds, reach out to our support team at <support@codegrade.com>.

<details>

<summary><strong>Appendix: cov_grade.sh</strong></summary>

The script below parses the JaCoCo XML report and outputs a line coverage grade as a fraction (e.g. `3/4`). CodeGrade uses this fraction to award rubric points proportionally. If LINE coverage data is not present in the report, the script falls back to INSTRUCTION coverage. If the XML cannot be parsed at all, it returns a grade of `0/1`.

```bash
#!/usr/bin/env bash
set -uo pipefail

COV_XML="${1:-cov.xml}"

# If FD 3 is open, write structured JSON there. Otherwise skip JSON output.
if { true >&3; } 2>/dev/null; then
  JSON_FD=3
else
  JSON_FD=""
fi

python3 - "$COV_XML" "${JSON_FD:-}" < 2 else ""

def emit_json(obj) -> None:
    if not json_fd:
        return
    data = (json.dumps(obj) + "\n").encode("utf-8")
    os.write(int(json_fd), data)

try:
    root = ET.parse(path).getroot()
except Exception:
    covered = 0
    missed = 1
    total = 1
    frac = Fraction(0, 1)

    print("Code Coverage Grade")
    print("-----------------------")
    print(f"Covered={covered}")
    print(f"Missed={missed}")
    print(f"Total={total}")
    print(f"Grade={frac.numerator}/{frac.denominator}")

    emit_json({
        "tag": "points",
        "points": f"{frac.numerator}/{frac.denominator}"
    })
    sys.exit(0)

covered = None
missed = None

for counter in root.findall("counter"):
    if counter.get("type") == "LINE":
        missed = int(counter.get("missed", "0"))
        covered = int(counter.get("covered", "0"))
        break

if covered is None:
    for counter in root.findall("counter"):
        if counter.get("type") == "INSTRUCTION":
            missed = int(counter.get("missed", "0"))
            covered = int(counter.get("covered", "0"))
            break

if covered is None:
    covered = 0
    missed = 1
    total = 1
    frac = Fraction(0, 1)
else:
    total = covered + missed
    frac = Fraction(covered, total) if total > 0 else Fraction(0, 1)

print("Code Coverage Grade")
print("-----------------------")
print(f"Covered={covered}")
print(f"Missed={missed}")
print(f"Total={total}")
print(f"Grade={frac.numerator}/{frac.denominator}")

emit_json({
    "tag": "points",
    "points": f"{frac.numerator}/{frac.denominator}"
})
PY

exit 0
```

{% hint style="info" %}
The script expresses the grade as an exact fraction (e.g. `7/10`) rather than a percentage. CodeGrade reads this fraction directly to calculate rubric points, so the output format must not be changed.

When `LINE` coverage data is absent from the report — which can happen with certain JVM configurations — the script automatically falls back to `INSTRUCTION` coverage. Both metrics are present in every standard JaCoCo XML report.

If the XML file cannot be parsed (for example because the JaCoCo agent failed to write `jacoco.exec`), the script returns `0/1` rather than exiting with an error, so the AutoTest run completes and the student receives a zero for the coverage category instead of a broken result.
{% endhint %}

</details>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.codegrade.com/automatic-grading-guides/java/grading-test-coverage-with-jacoco.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.