Running Docker

Grade a Dockerized web app

In this guide we explore how to grade a containerized web application with CodeGrade. Students submit a small web app together with a Dockerfile; CodeGrade builds the image, runs the container, and checks that the website actually responds over HTTP.

The same Setup + Tests pattern works for any Dockerized project — Node, Go, .NET, a database — not just the Flask example used here.

Consider the example submission below. It is a tiny Flask web server that listens on host 0.0.0.0, port 8080, and serves three endpoints:

Method & path	Response
GET /	Hello, CodeGrade!
GET /health	OK
GET /add/<a>/<b>	the sum of a and b

app.py

from flask import Flask

app = Flask(__name__)


@app.route("/")
def index():
    return "Hello, CodeGrade!\n"


@app.route("/health")
def health():
    return "OK\n"


@app.route("/add/<int:a>/<int:b>")
def add(a, b):
    return f"{a + b}\n"


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

requirements.txt

Dockerfile

Note: This guide drives Docker entirely from Bash Script blocks rather than a built-in language block, so it works regardless of the language inside the container.

AutoTest Setup

In the Setup phase we install Docker, start the daemon, and open the Docker socket so the (non-root) Tests phase can use Docker without sudo. The Setup phase has internet access and sudo rights, so this is the right place to do all privileged work.

Add a Script block to the Setup phase with the following script:

#!/usr/bin/env bash
# Install Docker, start the daemon, and open the socket so the (non-root)
# test steps can run `docker` WITHOUT sudo.
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive

# 1. Install Docker if it isn't already there.
#    docker-buildx silences the "legacy builder is deprecated" warning.
if ! command -v docker >/dev/null 2>&1; then
  sudo -n apt-get update
  sudo -n apt-get install -y --no-install-recommends docker.io docker-buildx || \
    sudo -n apt-get install -y --no-install-recommends docker.io
fi

# 2. Start the daemon (systemd if present, otherwise dockerd directly).
if command -v systemctl >/dev/null 2>&1 && sudo -n systemctl start docker 2>/dev/null; then
  :
else
  sudo -n sh -c 'dockerd >/tmp/dockerd.log 2>&1 &' || true
fi

# 3. Wait for the daemon socket to come up.
for _ in $(seq 1 30); do
  [ -S /var/run/docker.sock ] && sudo -n docker info >/dev/null 2>&1 && break
  sleep 1
done

# 4. Open the socket so the test step can talk to Docker without sudo.
sudo -n chmod 666 /var/run/docker.sock

sudo -n docker --version

Note — why chmod 666 the socket? The grading user isn't in the docker group, and group membership only takes effect on the next login — so it can't help within a single run. Opening the socket during Setup (where sudo works) lets the Tests phase run plain docker commands. This is safe for a throwaway grading sandbox.

AutoTest Tests

In the Tests phase we upload a small Test.sh script, build the student's image, run it, and check that the website responds. Configure the blocks as follows (the names in quotes match the screenshot below):

1. Add an Allow internet block as the outermost block. The build needs to pull the base image and install packages, and the Tests phase has no internet by default.

2. Inside it, add an Upload Files block ("Upload test") and upload the Test.sh script shown below.

3. Also inside Allow internet, add a Connect rubric block ("Running the App") and select your rubric category. This ties the pass/fail outcome to the grade.

4. Inside Connect rubric, add an IO Test block ("Running docker"). In its command field, copy the uploaded script into the working directory and run it:

   # Run the bash script and check the match blocks.
   cp $FIXTURES/*.* .
   ./Test.sh

5. Inside the IO Test, add a Substring Match block ("Checking the App"). Leave Input empty and set Expected in output to Hello, Web app is up!. Keep it case-sensitive and include whitespaces.

The Tests phase: an IO Test that runs Test.sh with a Substring Match, nested inside Connect rubric and Allow internet.

The Test.sh script does the actual work: it builds the image, runs the container, and prints Hello, Web app is up! when the site responds. The Substring Match looks for exactly that line, so a working submission passes and fills the rubric category.

#!/usr/bin/env bash
# Build the student's Dockerfile, run the container, and check the site is up.
# Prints the line the Substring Match block looks for. No sudo needed:
# the Setup step already opened the Docker socket.
set -euo pipefail

# Clean up any container left from a previous run.
docker rm -f cg-app >/dev/null 2>&1 || true
trap 'docker rm -f cg-app >/dev/null 2>&1 || true' EXIT

# 1. Build the student's image.
docker build -t cg-submission .

# 2. Run it in the background.
docker run -d --name cg-app cg-submission >/dev/null

# 3. Find the container's IP on the Docker network.
IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cg-app)

# 4. Wait up to 30s for the website to start responding.
for _ in $(seq 1 30); do
  curl -sf "http://${IP}:8080/" >/dev/null 2>&1 && break
  sleep 1
done

# 5. Print the marker when the site is up; otherwise show logs and fail.
if curl -sf "http://${IP}:8080/" >/dev/null 2>&1; then
  echo "Hello, Web app is up!"
else
  echo "Website is not responding on port 8080" >&2
  docker logs cg-app >&2 || true
  exit 1
fi

Note: The Upload Files command copies the fixtures into the working directory (cp $FIXTURES/*.* .) so Test.sh sits alongside the student's Dockerfile and app.py. Depending on your CodeGrade version this variable may be $UPLOADED_FILES — if Test.sh is "not found", check that first.

Tip: To grade a specific endpoint instead of just "is it up", change what Test.sh prints the marker for, e.g. only echo "Hello, Web app is up!" when curl .../add/3/5 returns 8.

Warning: The container listens on port 8080. If your students' apps use a different port, update both the Dockerfile (EXPOSE) and Test.sh. Reaching the container by its IP avoids host port-mapping conflicts when several submissions are graded in parallel.

Test and publish your AutoTests

Always test your configuration before releasing it to students.

1. Press Build snapshot at the bottom of the test block sidebar.

2. When prompted, upload the three reference files above as your Test submission.

3. Confirm the snapshot run fills the rubric category to 100% and that the output shows PASS: website is up.

4. Once you're happy, click Publish snapshot to make the tests available to students.

Note: Docker images and pip install can make the first build slow. If a snapshot times out, wrap the test Script in a Timeout each block and raise the limit (the default is 120 seconds).

Conclusion

You've built an assignment that grades a real, containerized web application end to end: CodeGrade installs Docker in Setup, builds the student's image, runs it, and verifies the site responds — all tied to a rubric. The same Setup + Script + Allow internet pattern extends to any Dockerized project.

For more on the blocks used here, see the AutoTest V2 Blocks guide, or reach out to our support team at [e-mail].