📒Jupyter Notebook

This guide explains how to configure automatic grading for Jupyter Notebook submissions using CodeGrade’s AutoTest v2 framework.

1. Submission Settings

To allow students to submit .ipynb files, ensure that the Editor is enabled in the Submission Settings of your assignment. You do not need to upload a template file at this stage.

2. Building the AutoTest

Setup Phase

In the Setup phase:

• Install any required dependencies (e.g., matplotlib, numpy, pandas) using pip.

• Upload any additional files needed during testing.

Test Phase

In the Test phase, begin by adding a Jupyter Notebook block.

Uploading a template file

Within the block, upload your template notebook. This will serve as the reference structure for testing, as well as a starting file for your students.

To generate a compatible template notebook with CodeGrade cell tagging:

1. Click on the uploaded notebook in the block to preview it.

1. In the pop-up window, download the file—this version contains the necessary cell tags for referencing individual cells in test blocks.

You can now fill in the answers in this downloaded notebook using your preferred editor. This filled in file will serve as the Test Submission.

Creating a template file from scratch

If you’d like to create a new full template file from within CodeGrade you can:

1. Click on "Create Empty" in the Jupyter Notebook block

2. Upload a test submission (it doesn’t have to be the actual test submission you will use later) and publish the snapshot

3. Enter Student View and create a new submission

1. You can build your Notebook from within the editor by adding and ordering cells. Submit this file and download it.

3. Internet Access in Jupyter Notebooks

By default, internet access is disabled whenever notebook code runs, both while your AutoTest tests run and while students run their own cells in the notebook editor. Internet is only enabled during the AutoTest Setup phase; after that, all notebook code executes without connectivity.

This means code that downloads a dataset, calls an external API, or runs pip install from within the notebook will fail during testing and when a student runs it in the editor — unless you explicitly allow it.

To grant internet access, wrap the Jupyter Notebook block inside an Allow Internet block:

1. In the Test phase, add an Allow Internet block.

2. Nest your Jupyter Notebook block (including its Run In Notebook blocks) inside it.

All blocks nested inside the Allow Internet block will run with internet access enabled. This applies to both AutoTest runs and student cell execution in the editor.

Where possible, install dependencies and prepare environment in the Setup phase instead of at test time. This keeps test runs faster and more reliable. For more information on the Allow Internet block, visit AutoTest V2 Blocks.

4. Inserting Test Code

To test specific cells in the student’s notebook, nest Run In Notebook blocks within the Jupyter Notebook block.

You can insert test code:

• At the start of the notebook

• At the end of the notebook

• Before or after a specific cell

In most use cases, inserting code after each question cell is ideal. For example, you may want to test the function a student writes with simple assertions.

To do this:

1. Choose “After a cell” from the insertion options.

1. Click on the "Cell" field to open the notebook preview.

1. Select the cell after which you want the test code inserted. Write your testing code in the Run In Notebook block.

1. Repeat this process for each question or notebook cell you wish to test.

5. Publishing the snapshot

With your Notebook blocks now set up, click the bottom right Build Snapshot button to build your AutoTest Snapshot. The first time you build a snapshot, you are required to upload a Test Submission; in this case it will be built for you from the template notebook you created. Simply click to generate a Test Submission:

Example: Testing plots

You can even test plots programmatically. Below is an example of assertions for a simple plot of y = x^2:

assert list(line.get_xdata()) == x, "X data does not match expected values"
assert list(line.get_ydata()) == y, "Y data does not match expected values"
assert ax.get_xlabel() == 'x', "X-axis label incorrect"
assert ax.get_ylabel() == 'y', "Y-axis label incorrect"
assert ax.get_title() == 'y = x^2', "Plot title incorrect"

These assertions verify that the correct data was plotted and the axis labels and title match expectations.