# CodeGrade Help Center Welcome to CodeGrade’s help center! We're here to answer your questions. Can't find what you're looking for? Send our support team a note at ! ## How to use CodeGrade? ### Use built-in content {% content-ref url="/pages/nWIFwxzxhYtZ3SEz2JON" %} [Community Library](/use-built-in-content/community-library) {% endcontent-ref %} {% content-ref url="/pages/MaFNQTC28PIj9a6zQc0T" %} [Introduction to Python Curriculum](/use-built-in-content/introduction-to-python-curriculum) {% endcontent-ref %} ### Or, create your own assignments {% tabs %} {% tab title="Step 1: Build" %} {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIAjb5fC1Bl7IxD3" %} [Add Rubric](/setup-assignment/build-assignment/setting-up-a-rubric) {% endcontent-ref %} {% content-ref url="/pages/-MLxJid2za9q19Ha5eMn" %} [Add Automatic Grading](/setup-assignment/build-assignment/creating-automatic-tests) {% endcontent-ref %} {% endtab %} {% tab title="Step 2: Grade" %} {% content-ref url="/pages/-MMKmqfIQu40QVNbqYEk" %} [Give Feedback](/setup-assignment/grade-your-assignment/grading-and-giving-feedback) {% endcontent-ref %} {% content-ref url="/pages/FeTdE94yW6RXlMw9KQZZ" %} [Give a Grade](/setup-assignment/grade-your-assignment/give-a-grade) {% endcontent-ref %} {% endtab %} {% tab title="Step 3: Analyze" %} {% content-ref url="/pages/-MMKlbVFRXtgnmFVbMee" %} [Detect Plagiarism](/setup-assignment/analyze-your-assignment/checking-for-plagiarism) {% endcontent-ref %} {% content-ref url="/pages/-MMKmkKRDOxpJM\_8Rzf7" %} [Analytics](/setup-assignment/analyze-your-assignment/viewing-analytics) {% endcontent-ref %} {% content-ref url="/pages/tmNsPYPE7e3WHBDPCGEI" %} [View and export grades](/setup-assignment/analyze-your-assignment/view-and-export-grades) {% endcontent-ref %} {% endtab %} {% tab title="Other features" %} {% content-ref url="/pages/-MMKlIaLtrLi8WydKWVb" %} [Peer Feedback](/setup-assignment/other-features/setting-up-peer-feedback) {% endcontent-ref %} {% content-ref url="/pages/-MMKlImjUnQ5KPfla\_7S" %} [Group Assignments](/setup-assignment/other-features/making-a-group-assignment) {% endcontent-ref %} {% content-ref url="/pages/-MMKlkccFKT2aJpsQ4r8" %} [Collaborative Grading](/setup-assignment/other-features/dividing-submissions-over-graders) {% endcontent-ref %} {% content-ref url="/pages/-MdX453UycGQL7OiUwem" %} [Anonymous Grading](/setup-assignment/other-features/grading-anonymously) {% endcontent-ref %} {% content-ref url="/pages/-MNZ-wkfulkIe1PlAxjo" %} [Manage Course Permissions](/setup-assignment/other-features/configuring-permissions-of-roles) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIkVJ8t9Gw5VpOUI" %} [Hand In Requirements](/setup-assignment/other-features/setting-up-hand-in-requirements) {% endcontent-ref %} {% endtab %} {% endtabs %} ## Automatic Grading Guides Click on any of the languages below. Is your programming language not listed, don't worry, CodeGrade supports any programming language. Contact us at and we'll help you set it up! {% content-ref url="/pages/jj72ts0v49Yks7MZ2EGa" %} [Create your first Python assignment](/automatic-grading-guides/python/create-your-first-python-assignment) {% endcontent-ref %} {% content-ref url="/pages/3SWN5XkkYLx5GarvlxWh" %} [Create your first Java assignment](/automatic-grading-guides/java/create-your-first-java-assignment) {% endcontent-ref %} {% content-ref url="/pages/4EqwFBrpNZTiL7CZm23G" %} [R](/automatic-grading-guides/r) {% endcontent-ref %} {% content-ref url="/pages/iSdvQe08LPiIovScq06T" %} [Create your first C assignment](/automatic-grading-guides/c/create-your-first-c-assignment) {% endcontent-ref %} {% content-ref url="/pages/KbSE5Rx8a1S22kb9Nepu" %} [Create your first C++ assignment](/automatic-grading-guides/c++/create-your-first-c++-assignment) {% endcontent-ref %} {% content-ref url="/pages/NuBzq0iJWL9zdaKN0pXT" %} [AutoTest V2 Blocks](/automatic-grading-guides/autotest-v2-blocks) {% endcontent-ref %} ### Guides for your students {% content-ref url="/pages/-MLxDxR4CdcXCo33A8gv" %} [For students](/for-students/getting-started) {% endcontent-ref %} ### Any other questions? Any other things you are looking for? Check out our Frequently Asked Questions section below! # FAQ ### General questions * [How do I use CodeGrade in Safari?](/faq/using-codegrade-in-safari) * [How do I create Sections?](/faq/using-sections) * [How do I set the roles of users in my course?](/faq/configuring-roles-of-members) * [How do I archive my course?](/faq/archiving-your-course) * [How do I add members to my (standalone) course?](/faq/adding-new-members) * [How do I release grades and feedback?](/faq/releasing-grades-and-feedback) * [How do I export submissions?](/faq/exporting-submissions) * [How do I change the grading scale?](/faq/choosing-a-grading-scale) * [How do I create course-wide feedback snippets?](/faq/creating-course-wide-snippets) * [How do I view the course gradebook?](/faq/course-gradebook) * [How do I allow students to hand in after the deadline?](/faq/allow-students-to-hand-in-after-deadline) * [How do I personalise my CodeGrade app?](/faq/accessibility-and-personalization-settings) # Using CodeGrade in Safari CodeGrade is often used as a third-party tool in LMSs such as Canvas. CodeGrade requires third-party cookies to be enabled in your browser in order for it to send and receive information from the LMS. When using Safari, you might encounter this error when viewing a CodeGrade assignment as an iframe within the LMS:
This error is caused because Safari's default privacy settings prevents third-party cookies which CodeGrade needs in order to collect and send data to your LMS system. This issue can be resolved by following these steps: #### Enable cross-site tracking 1. Open Safari on your Mac. 2. Choose "Safari" > "Settings" in the Apple toolbar at the top of your screen. 3. Navigate to the "Privacy" tab. 4. Disable the "Prevent cross-site tracking" option by **unchecking** the box beside it.
#### Enable cookies 1. Navigate to the "Advanced" tab. 2. Disable the “Block all cookies” option by **unchecking** the box beside it.
# Using sections With Sections, you can divide students and graders in different groups. This allows you to have certain graders only be able to see and grade certain students if they belong to the section this grader belongs to. ## How to create Sections?
1. Navigate to the Course Settings Page and then to Sections. 2. Create your Sections by filling in the Name and selecting Members. These members should be both the students and the graders within this section. ## Setting Section permissions
If you want to set it up so TAs (or any other role) can only grade and view students within their own Section, you have to update the permissions. 1. Go to the Permissions tab on the Course Settings page. 2. Uncheck the following permissions for TAs 1. Edit groups of others 2. Edit submissions by others 3. View groups of others 4. View submissions by others 3. Now TAs can only view and grade submissions that are part of students in their Section. ## Different deadlines per Section With CodeGrade's Schedule feature, it is possible to set up different submission schedule settings per Section. You can individually set up the Availability, Deadline and Lock date per assignment per Section. To edit these settings for an assignment: 1. Go to the General tab of the Assignment Settings page of the assignment. 2. Find the "Schedule" section. 3. Press the "+ Add" button to add a new Schedule: 1. Select the Section(s) this new Schedule should apply for. 2. Set the Availability, Deadline and Lock date that should apply for this Section. 4. Repeat this process for all Sections you want to set up Schedules for.

Setting up individual Schedule settings per Section.

{% hint style="info" %} If a student is in multiple Schedules, the most specific one applies first (e.g. information will first be taken from a specific Section over the "Everyone" Schedule). If a student is in multiple Sections, the most lenient is taken for each individual setting (i.e. most lenient availability setting, most lenient deadline and most lenient lock date). {% endhint %} # Configuring roles of members All members in your CodeGrade course have a role, which you can manually change. CodeGrade has an extensive roles and permission system, which allows you to customize your course exactly how you want. You can change the roles of members and create new roles. {% hint style="success" %} It is a good practice to confirm the correctness of the roles of the members that are synchronized from your Learning Management System. These roles will be correctly mapped in most occasions, but will sometimes map wrong for custom LMS roles. {% endhint %} To change roles, follow the instructions below: ![Changing the role of a member in the Course Manager](/files/I2ftzzF2pfMl2VsQWlSl) 1. Navigate to the **Course Settings Page** and to the *Members* tab. Here you can find a list of all members that are currently enrolled in your course. 2. Find the member(s) you want to change roles for. Use the search bar to find a specific member or sort on full names or roles to find a group of members easily. 3. Use the dropdown to select a new role for a member. Changes take effect immediately. #### Creating a new role: In addition to our default roles, you can also create your own custom roles: ![Creating a new role in the Course Manager](/files/gPps4qUIgFMZCtGsBwip) 1. On the **Course Settings** page of your course navigate now to the *Permissions* tab. 2. Scroll down and add a new role by typing in the name and clicking **Add**. 3. By default, this new role will automatically have the same permissions as a regular student role. You can now [change the permissions](/setup-assignment/other-features/configuring-permissions-of-roles) of this role. {% hint style="success" %} A common custom role is the "**Late Student"** role, which is given to a subset of students who are allowed to hand in after the deadline. Find out how to do that below: :point\_down: {% endhint %} {% content-ref url="/pages/-MN-A7QLTJ2k44qHcAT8" %} [Allowing students to hand in after the deadline](/faq/allow-students-to-hand-in-after-deadline) {% endcontent-ref %} # Archiving your course After finishing your course, you can archive it. You can archive your course after you have finished it. Archiving a course will hide it for students, and will sort it below all non-archived courses. You can still import data from archived courses. Archiving your course can make it easier to differentiate between different iterations of the same course. {% hint style="success" %} Archiving your course does not result in loss of data and can be reversed at any time. {% endhint %} ![Archiving your course in the Course Manager](/files/hwODMppywflX0uRdsiGJ) 1. Navigate to the **Course Settings Page** and to the *General* tab. 2. Enter the ***Danger Zone*** and press the red **Archive Course** button. 3. After confirming, your course is successfully archived and invisible to students. {% hint style="warning" %} Archiving your course makes it invisible to students. They will thus also not be able to view their feedback or submissions anymore. {% endhint %} # Adding new members If you are using CodeGrade without a Learning Management System, you can add new members to your course in multiple ways! {% hint style="warning" %} This guide only applies to you if you are did not integrate CodeGrade with your Learning Management System (via LTI). If you did: new members are automatically synchronized from your LMS. {% endhint %} You can invite new members to join your course by bulk enrolling them (from a CSV) or by sending them Course Enroll Links. ## Bulk enrolling new members If you have a list with the information of the members to join your course, you can use CodeGrade's bulk enroll feature to add them to your course. CodeGrade requires three fields of information per user: their username, email and full name. After enrolling members and choosing their role in your course, new members will automatically get an email with a link to reset their password.

On the Course Management Page, select Members and navigate to "Bulk Enroll".

1. Navigate to the **Course Settings Page** and to the *Members* tab. 2. Here you can find the *Bulk Enroll* feature, navigate there and select a `.csv` file containing a list of the members you want to enroll. \ *Your `.csv` file may contain multiple columns, but should at least contain an email address, full name and username per member (it is possible to reuse the email address as the member's username). Superfluous columns are ignored and column names and order can be arbitrary.* 3. After uploading: select the correct columns containing the username, email address and full name and confirm if the displayed member list is as expected. 4. Select the role of the members you want to enroll, this role applies to all the members you are enrolling. You can always [manually change roles of users](/faq/configuring-roles-of-members) afterwards. 5. Press **"Submit"** to enroll all the members. Members with an existing account will be added to your course, completely new members will receive an email to set up their password.

After uploading your .csv file, select the correct columns and role and press "Submit" to bulk enroll the members.

## Inviting members using an Enroll Link You can also invite new members to join your course using CodeGrade's Course Enroll links. You can share these links with your students or colleagues so that they can join your course. These links allow users to enroll in your course by logging in to their existing CodeGrade account, creating a new CodeGrade account or using a SSO tool to login. 1. Navigate to the **Course Management Page** and to the *Members* tab. 2. Here you can find your current *Course Enroll Links* or create a new one. Click the **Add New** button to create a new Course Enroll link for your course. 3. Set an *Expiration Date*, after which the Course Enroll link will become invalid. 4. Select the *Role* that users using this link will sign up as (e.g. sign up as a student), you can always [manually change roles of users](/faq/configuring-roles-of-members) afterwards. 5. Toggle if users are only able to join your course with an existing CodeGrade account or are allowed to register a new account for your course too. 6. Press **Save** to save your Enroll Link, copy the link to clipboard an share it with your students or colleagues.

After setting up a Course Enroll Link, copy the link to share with students or colleagues.

# Releasing grades and feedback Learn how to release grades and feedback to your students and pass back these grades to the gradebook in any LMS, including Canvas, Blackboard, Moodle, Brightspace. Releasing grades and feedback to students (and sending them back to your LMS) is very straightforward: 1. Go to the **Assignment Settings**. 2. Navigate to the **General tab** and find the **Grades Published** setting. 3. Set this toggle to to *"Yes"* and press **Submit**. 4. All grades are now automatically sent back to Canvas, Blackboard, Moodle or Brightspace and feedback (including peer feedback and responses to student questions) is visible to students.

Toggle the "Grades published" option to "Yes" to publish grades and feedback to students and the LMS.

{% hint style="success" %} While the assignment's grades are published, all grades and changes to grades are immediately sent back to Canvas, Blackboard, Moodle or Brightspace. {% endhint %} {% hint style="warning" %} Publishing grades for an assignment that is not available yet may make the grades show up in your LMS' grade book. {% endhint %} {% hint style="info" %} **Want to force refresh grades in your LMS?** First toggle back to *"No"* press Submit and then toggle back to *"Yes"* and Submit again. This should force a resync. {% endhint %} # Exporting Submissions There are two ways in which you can get data out of CodeGrade. Using the assignment export or by using our API. ## Using the assignment export You can easily export all submissions and all grade and feedback data of an assignment. You can also filter the export using search or by assigned user. ![Downloading all submissions in CodeGrade](/files/XeM69X0TZ2j6VfbicJaw) 1. Navigate to the **Assignment Home** page of the assignment you want to download submissions for. 2. Navigate to the **"Export" tab**. 3. Click "**Export files**" to download an archive containing all submissions or "**Export Metadata**" to export the grade and feedback data to a CSV. ## Using the API You can easily create extensible API scripts to get any sort of data out of CodeGrade. CodeGrade comes with a Python package so you can easily write scripts. [Click here to go to the Python API documentation.](https://python.api.codegrade.com/) # Choosing a grading scale Educational institutions worldwide have various grading scales. CodeGrade supports two types of grading scales: points and percentages. CodeGrade offers two types of grading scales, to support as many educational workflows as possible. Out of the box, CodeGrade assignments can have a points-based grading scale or percentage-based grading scale: * **Points-based grading scale**: students can score any (decimal) number between 0 and 10 inclusive. With a 0 being the lowest grade and 10 the highest. * **Percentage-based grading scale**: students can score any number between 0% and 100% percentage. With 0% being the lowest grade and 100% the highest. {% hint style="success" %} **Using another custom grading scale in your LMS?** CodeGrade sends a universal grade to your LMS, which will be automatically converted to the custom grading scale of your LMS and show up like that in the gradebook of your LMS. Switching your grading scale in CodeGrade **only** changes the grading scale in the user interface of CodeGrade. {% endhint %} You can choose a grading scale for each assignment in CodeGrade by following the steps below: ![Updating the grading scale for your CodeGrade assignment](/files/AEDzczFIvFkse32LO8vU) 1. Navigate to the **Assignment Settings page** of the assignment you want to update the grading scale for. 2. Go to the **General** tab and navigate to the **Grading scale** option under the *"Advanced options" in "General settings"* section. 3. Toggle to **Points** or **Percentage**. Press **Submit** to submit your changes. # Creating course-wide snippets Course-wide snippets are linked to a course and shared by all graders in that course, only available in a specific course. Course-wide snippets are available for a specific course and are automatically available to all graders in all assignments of that course. {% hint style="success" %} Next to making grading more efficient for you and your graders, creating course-wide snippets forces you to think about common and expected errors and mistakes during your assignments, which is always a good practice. {% endhint %} Create new snippets in your course by following these steps: ![Creating new course-wide snippets](/files/QQDdf0twoawfVstt7k0M) 1. Choose any course and click on the **Course Settings** button present on the right of the screen. 2. Go to the **Snippets** tab and click on the **New snippet** button (+ symbol). 3. Enter a *name* and the *replacement text* for the snippet in the respective input fields, and press **Save**. This snippet can now be used anywhere throughout this particular course. {% hint style="success" %} Personal snippets and course-wide snippets can have the same keywords, these can then be used and selected next to each other. {% endhint %} # Course Gradebook ## View and export grades of an entire course You can find an overview of all grades of all assignments in a course in the gradebook. This gradebook shows a full list of all members, so it will also show the students that did not make any submissions (yet).

Viewing the gradebook for a course in CodeGrade and downloading a CSV with all grades.

1. Navigate to the **Course Settings Page** of the course you want to find or export all grades for. 2. Navigate to the **"Gradebook" tab**. 3. You will find a list of all students, grades and assignments on this page. Click **"Download CSV"** to download a CSV with all grades. # Allowing students to hand in after the deadline Allow all students to hand in after the deadline, or give a subset of students a deadline extension manually. Deadlines in CodeGrade are hard by default, meaning that students cannot hand in after the deadline anymore. Sometimes, you may want to allow all your students to hand in after the deadline or give a small subset of your students a deadline extension manually. Depending on your needs, CodeGrade offers two methods to allow this: * Allow all students to submit after the deadline by giving them a course-wide permission. * Give a subset of students a deadline extension on a per-assignment level. ## 1. Allow all students to hand in after the deadline with a course-wide permission: 1. Navigate to the "**Course Management**" page of your course. 2. Go to the "**Permissions**" tab and search for the "**Create submissions after the deadline"** permission. 3. Enable this permission for students by checking the box in the Student role column. Students can now still hand in after the deadline. Late submissions via permissions will always be visibly marked, so you can easily differentiate them. {% hint style="info" %} Students can **not** hand in after the lock date using the permission explained in this guide, meaning the lock date can be seen as a *second deadline*. To allow students to also hand in after the lock date, give them the permission '*Upload after lock date'* too. {% endhint %} ![Giving students the "Upload after deadline" permission in the Course Manager](/files/YRcmyKedFmT6vKmWg17f) ## 2. Give a subset of students a deadline extension for an assignment 1. Create a Section with the subset of students that are given a deadline extension in your course. Call this Section (for example) "Deadline Extension". [Click here to learn how to set up a Section in your course!](/faq/using-sections) 2. Navigate to the General tab of the Assignment Management Page for the assignment you want to give students a deadline extension for. 3. In the Schedule section, add a new Schedule by clicking the "+ Add" button. 4. In this new Schedule, select the just created "Deadline Extension" section in the "For" field. 5. Fill in the remaining settings for your extended deadline, including the Availability, Deadline and Lock Date settings. 6. Press Submit to save your extension. The students in this Section will now have different Schedule settings (e.g. deadline and availability) than other students. As they simply have a different deadline, their submissions will not be marked as late. You can set up multiple extensions for different Sections per assignment.

Extra Schedule settings for students with a Deadline Extension.

# Accessibility and Personalization Settings CodeGrade provides several accessibility and personalization options that allow users to customize how the interface and code viewer appear. These settings help create a more comfortable working environment and can be adjusted at any time from the **User Profile** icon located on the left-hand side of the app.

Configuring preferences in CodeGrade

{% hint style="success" %} CodeGrade includes built-in support for screen readers, ensuring that essential parts of the interface can be accessed using assistive technologies. These features do not need to be turned on or configured separately—the platform is designed to be accessible by default from the moment you log in. {% endhint %} *** ### Personalization Settings Overview Users can tailor CodeGrade to their personal preferences, and all settings are saved to their account so they persist across sessions. Accessibility and personalization options can be updated at any time through the user profile menu. To update these settings:\ Click the **User Profile** → **Preferences** → choose your desired options. | Setting | Options / Values | Default | Description | | ---------------------------- | ---------------------------------------------- | --------------- | --------------------------------------------------------------------------------------------- | | **Show Whitespace** | On / Off | On | Toggles visibility of whitespace characters such as spaces, tabs, and newlines in code views. | | **Code Font Size** | Pixel value (e.g., 12, 14, 16) | 12px | Sets the font size used in code blocks. | | **Theme** | Light, Dark, System | Light | Controls the appearance of the CodeGrade interface. | | **Submission Tab Selection** | Smart, Code, Feedback, AutoTest, Peer Feedback | Smart Selection | Determines which tab opens first when viewing a submission. | > CodeGrade strives to ensure that every student and instructor can fully engage with coding assessments and feedback — regardless of their device, preferences, or accessibility needs. {% hint style="info" %} Note: Additional information, including access to our VPAT can be requested [here](https://security.codegrade.com/resources). {% endhint %} # Community Library
A great way to start with setting up your assignment is by starting out with a Community Library assignment. Once you have created an assignment (see [Create assignment](/setup-assignment/build-assignment/creating-an-assignment)), you can import a Community Library assignment as follows: 1. Go to the General settings of the assignment by pressing the Gear icon in the top right of the screen. 2. Scroll all the way to the bottom to the "Danger Zone" 3. Press Community Library 4. Select the programming language you are teaching 5. Click on any assignment to preview it and then press import to load the content into your assignment. # Introduction to Python Curriculum The CodeGrade Introduction to Python course is a basic Python course with 6 lessons. Students are not required to have any prior knowledge on programming or Python. This course will cover the basics of programming in Python, modules like NumPy and Pandas, OOP and introduce students into the fields of data science and artificial intelligence. Each lesson covers multiple topics, which are assessed in fully autograded assignments. Students are encouraged to continuously hand in to these assignments to test out their implementation. Automatic checks are built in to detect whether students have put the topics explained in that week into practice. Students can use CodeGrade’s built-in editor to work on the assignments, which is especially recommended for the first couple of weeks. Templates are available and set up for students in the editor. This course is ready to use and can be taught as a whole, or can be cherrypicked from to fit in your Python course. **Summary** * 30+ ready to use Python assignments, including: * Assignment description; * CodeGrade assignment + AutoTest; * Solution code; * Template code and required fixtures. * All available through the Community Library. ## Overview of the curriculum * **Lesson 1:** print, variables, arithmetics and data types (6 assignments). * **Lesson 2:** Input, conditionals and strings (7 assignments). * **Lesson 3:** Iterations (for- and while-loops) and functions (5 assignments). * **Lesson 4:** Data Structures (Lists, Dictionaries, sets) and recursion (8 assignments). * **Lesson 5:** Built-in Python Modules and basic Data Science using NumPy (4 assignments). * **Lesson 6:** Object Oriented Programming (4 assignments). ## Adopt All Introduction to Python assignments are available through the Community Library. ### Step 1: **Create empty assignments** For each assignment you want to use, you have to create an empty CodeGrade assignment. Either in your LMS or standalone. Follow this guide: [Create assignment](/setup-assignment/build-assignment/creating-an-assignment). ### Step 2: Import from Community Library The assignments will be available in the Community Library. You can find more information on how to do that here: [Community Library](/use-built-in-content/community-library). ### Step 3: **Tweaking the assignments** Some common tweaks that you may consider for your course, depending on your learning goals: * **Grades published:** toggle grades published to “Yes” so that students can see all instant automatic feedback and grades are automatically sent to your LMS. * **Toggling the editor:** by default, students have access to CodeGrade’s built-in editor for all assignments (and are presented with a template to start with). If you wish, you can toggle this and only allow uploading files. * **Deadlines:** a deadline is required for each assignment. If your course is self-paced, it is recommended to set a deadline in the far future to simulate that in CodeGrade. Deadlines are set from within your LMS. * **Rubric and test descriptions:** pre-made rubrics and test descriptions are available for all assignments. You may customize these in any way you want to better reflect your learning style and goals or to make certain elements more clear to students. * **Distributing templates and required files:** some assignments require additional (input) files for the students (e.g. a `.csv` dataset or a `.txt` input file for lexical analysis). These files are automatically provided as template in CodeGrade if the editor is turned on. It is a good idea to share these files via the LMS with your students too. ## **Support** We are here to make your course a success! Reach out to use at . # Build your assignment In four easy steps you can build your own assignment in CodeGrade. {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIAjb5fC1Bl7IxD3" %} [Add Rubric](/setup-assignment/build-assignment/setting-up-a-rubric) {% endcontent-ref %} {% content-ref url="/pages/-MLxJid2za9q19Ha5eMn" %} [Add Automatic Grading](/setup-assignment/build-assignment/creating-automatic-tests) {% endcontent-ref %} Alternatively, you can start by importing a previous assignment after you create an assignment: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} {% content-ref url="/pages/-MdHzkWW4tP\_gHD6EmVA" %} [Importing a previous assignment](/setup-assignment/build-assignment/creating-an-assignment/importing-a-previous-assignment) {% endcontent-ref %} # Create assignment ## Let's create an assignment, what CodeGrade version are you using? Select the Learning Management System you work in below or select Standalone if you are not using CodeGrade integrated with an LMS. {% content-ref url="/pages/-MLxNYYmP-T34ATfeKcr" %} [Standalone](/setup-assignment/build-assignment/creating-an-assignment/standalone) {% endcontent-ref %} {% content-ref url="/pages/-MLxNKC\_SL1TF9HgMG3c" %} [In Blackboard](/setup-assignment/build-assignment/creating-an-assignment/in-blackboard) {% endcontent-ref %} {% content-ref url="/pages/-MLxNSu8rUWq7Of9f0Er" %} [In Brightspace](/setup-assignment/build-assignment/creating-an-assignment/in-brightspace) {% endcontent-ref %} {% content-ref url="/pages/-MLxNSpJ3mTSsZsXXjYc" %} [In Canvas](/setup-assignment/build-assignment/creating-an-assignment/in-canvas) {% endcontent-ref %} {% content-ref url="/pages/-MLxNSmbUNK794ZrcBhb" %} [In Moodle](/setup-assignment/build-assignment/creating-an-assignment/in-moodle) {% endcontent-ref %} {% content-ref url="/pages/-MLxNSd6hcSBtN-ZOLEG" %} [In Sakai](/setup-assignment/build-assignment/creating-an-assignment/in-sakai) {% endcontent-ref %} {% content-ref url="/pages/-MLxN\_PvDUJmrPl3SzwK" %} [In Open edX](/setup-assignment/build-assignment/creating-an-assignment/in-open-edx) {% endcontent-ref %} # Standalone Learn how to create a new CodeGrade assignment for your course in CodeGrade Standalone. {% hint style="info" %} Use the :heavy\_plus\_sign: **Create Assignment** button on Course Home page to quickly create a new assignment. {% endhint %}
{% hint style="success" %} You should have been added to the correct course by the CodeGrade Support Team, if not, please[ contact us.](https://www.codegrade.com/contact) {% endhint %} You can also follow the steps below to create a new CodeGrade assignment on your standalone course: 1. Access your course, then navigate to **Course Settings** via the right-hand navigation pane or the dropdown menu beside the course name in the top-left corner. 2. In the *Create Assignment* section, enter a name for the assignment and click **Create**. 3. The **Assignment Settings** page for the new assignment will then open. We are now ready to setup the general settings in the assignment, click the link below to find out the possibilities! {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Blackboard To create a CodeGrade assignment in Blackboard, first pick which Blackboard user interface you're using, either Ultra or Original. ### Ultra course view 1. Press the small "**+**" button to add content and press **"Content Market".** 2. Under "**Institution Tools**" you will see CodeGrade. 3. Under "*CodeGrade*" press the small round "**+**" button in the bottom right. 4. A new assignment with name "*CodeGrade*" will now be added. 5. On the right, press on the 3 dots to edit this assignment. 6. Here you can set a name and add a "*Gradebook entry*" to make this a graded assignment with a due date. 7. After saving, click on the newly created assignment and CodeGrade will open. ### Original course view 1. Hover over **"Build Content"**. 2. Under **"Create"**, select **"CodeGrade"**. 3. Add a name and optionally a description for your assignment. 4. Set **"Points Possible"** to something greater than 0, e.g. 10. 5. Set a due date 6. Click **"Submit"** to create the assignment. 7. Click on the newly created assignment and CodeGrade will open. {% hint style="danger" %} If CodeGrade was integrated using the LTI 1.1 standard, do not set a due date in Blackboard, as this will be managed via CodeGrade. {% endhint %} {% hint style="success" %} [Grades are automatically sent back to Blackboard after setting the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Blackboard. {% endhint %} After creating the assignment in Blackboard, we can go ahead and open it and setup the General Settings: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Brightspace Follow the steps below to create a new CodeGrade assignment in Brightspace: 1. Navigate to the homepage of the course to create the assignment in. 2. Click the **"Content"** button at the top of the page. 3. In the **"Add Existing Activities"** dropdown, click *"CodeGrade"* to open a new dialog. 4. In the new CodeGrade dialog, provide the name and deadline of the assignment and press **"Create assignment".** You can optionally also set the "Available at" and "Locked at" dates. Making it a graded assignment: 1. Click on the CodeGrade assignment to open it (if it opens in a new tab, open the Brightspace tab to the left of it). 2. In Brightspace, you now see the possibility to add a Grade item. ([Learn more about Grade Items in Brightspace (D2L documentation)](https://documentation.brightspace.com/EN/le/grades/instructor/creating_grade_items.htm) 3. After adding a Grade item, relaunch the CodeGrade assignment and it now is a graded assignment! {% hint style="success" %} Grades are automatically sent back to Brightspace after setting [the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Brightspace. {% endhint %} Next up, we can set the General settings of the CodeGrade assignment: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Canvas Follow the steps below to create a new CodeGrade assignment in Canvas: 1. Click the **"+ Assignment"** button in Canvas. 2. Under submission type, select **"External Tool"**. 3. Under **"Enter or find an External Tool URL"** click the **"Find"** button. 4. In the dialog, select **"CodeGrade"** and click **"select"**. 5. Add your Due date and other Canvas options and finally save your assignment. 6. Your assignment will automatically open in Canvas and the CodeGrade environment will be visible. {% hint style="success" %} Grades are automatically sent back to Canvas after setting [the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Canvas. {% endhint %} Now, go ahead and set up the General Settings: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Moodle Learn how to create a new CodeGrade assignment for your course in Moodle. Follow the steps below to create a new CodeGrade assignment in Moodle: 1. In your Moodle course, press the "**+ Add an activity or resource**" button in the place you want to add your CodeGrade assignment. 2. From the window that now shows up, select "*CodeGrade*" and press "**Add**". 3. Now set the name for your assignment and edit additional settings. 4. Press "**Save**" again to save and display your new CodeGrade assignment. {% hint style="success" %} Grades are automatically sent back to Moodle after setting [the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Moodle. Only after this is done, the CodeGrade assignment will show up in the grade book. {% endhint %} Now, go ahead and set up the General Settings: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Sakai Creating a new CodeGrade assignment from within Sakai will correctly add this assignment to the corresponding course in CodeGrade, a new course will automatically be created in CodeGrade if it does not yet exist in CodeGrade. Follow the steps below to create a new CodeGrade assignment in Sakai: 1. In your Sakai course, navigate to the "**Resources**" page, which can be found in the sidebar. 2. Click on the "**Add Content**" button and select "*Add External Tool*" in the list that pops up. 3. In the External Tool Selector, select "*CodeGrade*". 4. Now, you can provide the *Tool Title* (name of the assignment) and *Button Text* and press "**Save**". {% hint style="success" %} A good practice is to keep the Tool Title and Button Text the same! {% endhint %} {% hint style="success" %} Grades are automatically sent back to Sakai after setting [the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Sakai. Only after this is done, the CodeGrade assignment will show up in the gradebook. {% endhint %} Now, go ahead and set up the General Settings: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # In Open edX Follow the steps below to create a new CodeGrade assignment in Open edX: 1. Edit the unit in which you want to add CodeGrade and select **Advanced** from the **Add New Component** section. Select *LTI Consumer*. 2. Select **Edit** in the component that appears. This should open a modal in which you can set up the CodeGrade assignment. 3. Set the **Display Name** to the name of the assignment. 4. The value for **LTI ID** depends on the configuration that your administrator set, however this will probably be `codegrade`. 5. The **LTI URL** is `https://app.codegra.de/api/v1/lti/launch/1` for most CodeGrade instances, if you use a custom CodeGrade URL replace `app` with your custom URL prefix. 6. Now scroll all the way to the bottom and make sure the following options are all set to **True**: * Scored, this option allows to passback a grade for the assignment * Request user’s email * Request user's full name * Request user’s username 7. Click the save button. {% hint style="success" %} Grades are automatically sent back to Open edX after setting [the **assignment state** to **Done** in CodeGrade.](/faq/releasing-grades-and-feedback) While the assignment is in the **Done** state, all grades and changes to grades are immediately sent back to Open edX. {% endhint %} Now, go ahead and set up the General Settings: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # Importing a previous assignment Most of the times, it is not necessary to set up a whole assignment from scratch. In CodeGrade, you can simply import / copy a previous assignment's settings to your current assignment. {% hint style="success" %} For the D2L Brightspace learning management system, using the built in "Duplicate Assignment" option will automatically import all assignment settings in CodeGrade correctly too. {% endhint %}

Importing an assignment on the Assignment Manager

1. Navigate to the **Assignment Settings** page and go to the **General** tab. 2. Scroll all the way down to the "*Danger Zone*", here you can find the *Copy Assignment* option. 3. First select the course from which you want to import an assignment. Secondly, select an assignment from the list of assignments in that course. 4. Press **Import**. All assignment settings will be copied to the current assignment now (including AutoTest, rubric, hand in requirements, submission settings, test submission, etc.). # General settings The General settings page allows you to schedule your assignment, determine how your students should submit their work, and publish the grades and feedback to students. To access the General settings, click on the gear icon at the top-right-hand side of the submission overview page. {% hint style="success" %} Don't forget to save your changes! Settings with unsaved changes now show an indicator, so you can easily spot when you still need to click the "Save" button. {% endhint %}

Assignment Settings Page for assignment

## General settings In the General settings options, you can adjust the following options: * **Name:** Adjust the name of your assignment. * **Type:** Make your assignment a "Normal" assignment or an "Exam". Exam mode offers some convenient features for hosting exams in a controlled environment. * **Grades published:** Determine whether students can see their grades or any manual feedback you have provided to them. Setting the Grades Published toggle to "Yes" will also sync students' grades with your LMS. If you created your assignment via an LMS, the name and type of assignment will automatically be copied and any adjustments must be made via your LMS. ## Schedules Using Schedules you can set the availability, deadline, and lock date of an assignment. By default, the first schedule you set up affects everyone. However, you can also set individual schedules for different sections of your course. For more information about how to create sections, see [Using Sections](/faq/using-sections). If you created your assignment via an LMS, the schedule will automatically be copied to CodeGrade. However, certain scheduling options cannot be transmitted to CodeGrade. For example, you cannot set schedules for individual sections via an LMS. This must be done in CodeGrade. By creating a new schedule, you can override the settings copied from your LMS. #### Availability The availability of your assignment affects whether your students can see and interact with your assignment. There are three options: * **Available at:** you can set an "available at" date and time, at which the assignment will automatically become available and visible to your students. * **Unavailable:** set your assignment to "unavailable" to make it invisible to your students. * **Always Available:** set your assignment to "always available" to make it indefinitely available to your students. #### Deadline and Lock Date The deadline and lock date settings allow you to specify when your students can no longer hand in work to the assignment. CodeGrade uses your system's timezone for both settings when setting dates and times. In most cases, a Deadline will suffice as the endpoint for an assignment. But, you can also set a Lock Date which can serve as an extended deadline. Submissions that are handed in after the deadline will be shown in red. Be aware that your students cannot automatically submit after the deadline, even if you have set a lock date. However, if you have set a lock date, CodeGrade will show a banner above the Schedule settings that you can click on to make this adjustment to your students' permissions automatically. If the deadline and lock date fields are greyed out, your assignment is linked to your learning management system and these settings should be set there. {% hint style="success" %} **Want an ongoing formative assignment?** Simply set no deadline and students can always hand in to your assignment! {% endhint %} ## Submission Settings The Submission settings allow you to select the ways you want students to submit code. There are three options which can all be enabled simultaneously. * **File uploader:** This option allows students to upload files from their computers. Students can also upload archives and they will be automatically extracted. * **GitHub/GitLab:** This option allows students to connect a GitHub or GitLab repository to the assignment. You can also provide a Template Repository that students will clone and use. * **Editor:** This option allows students to code directly in CodeGrade. You can also provide template files, which will be the starting point for students using the editor. **(Optional) Limit copy-pasting in the Editor** With the Editor enabled, you can use the **Allow copy-pasting** slider to set an upper limit on how much text students can paste at once, from **Unlimited** down to blocking pasting completely. This discourages students from pasting in code they didn't write. Staff can always copy-paste in the editor. For more information, see [Copy-Paste Limit in Editor](/setup-assignment/other-features/copy-paste-limit-in-editor). **Limiting Submission Attempts** {% hint style="info" %} If you want students **only** to use the Online Editor, you can enable the **Simple Submission Mode** under the General Settings Panel. When opening the assignments, students will then be directly brought to the Editor. {% endhint %}
#### (Optional) Limit the amount of submissions In addition to the submission methods, you can choose to limit the number of submissions your students can make in two ways: * **Maximum total amount of submissions:** This allows you to limit the maximum number of submissions students can make. By default, students can make unlimited submissions. * **Cool-off period:** This allows you to limit how frequently submissions can be made. For example, you could limit students to 2 submissions every 10 minutes. By default, students can submit as often as they like. ## Upload Test Submission With the Upload Submission option, you can upload a test submission. Having a test submission is necessary for testing your automatic tests. Just upload your solution file and check the "Test Submission" option before pressing "Submit" ## Add Rubric Next, add a rubric to help with grading your students' submissions! {% content-ref url="/pages/-MMKlIAjb5fC1Bl7IxD3" %} [Add Rubric](/setup-assignment/build-assignment/setting-up-a-rubric) {% endcontent-ref %} ## Optional tools The general settings have some optional tools available that you can use. {% content-ref url="/pages/-MMKlIkVJ8t9Gw5VpOUI" %} [Hand In Requirements](/setup-assignment/other-features/setting-up-hand-in-requirements) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIaLtrLi8WydKWVb" %} [Peer Feedback](/setup-assignment/other-features/setting-up-peer-feedback) {% endcontent-ref %} {% content-ref url="/pages/-MMKlImjUnQ5KPfla\_7S" %} [Group Assignments](/setup-assignment/other-features/making-a-group-assignment) {% endcontent-ref %} {% content-ref url="/pages/-MdX453UycGQL7OiUwem" %} [Anonymous Grading](/setup-assignment/other-features/grading-anonymously) {% endcontent-ref %} # Add Rubric The next step in setting up your assignment is adding a Rubric. Rubrics are used to award points to students either manually or with automatic tests. The points can be structured using different Rubric Categories. To create your Rubric, navigate to the Rubric tab and press "Create new rubric". Alternatively, use the "Copy a rubric" option to copy a rubric from another assignment you have created. ![Creating a new rubric for your CodeGrade assignment](/files/2nh4djj7JGmRdafZujAK) CodeGrade offers two types of Rubric categories: * **Discrete:** Using a Discrete rubric category, you can award points in discrete steps or "bins". These are great for "Pass/Fail" automatic tests or for manual grading with more options. * **Continuous:** Using a Continuous rubric category, you can award points on a scale. The scale is determined by setting an upper and lower bound of points. The upper and lower bound can also be negative. These are great for grading automatic tests as the number of points awarded scales according to the number of tests passed. For each rubric category, you can set a name and a description. These can be used to help make the grading requirements clearer to students. You can also reorder rubric categories by grabbing the handles on the left and dragging them into your preferred order. ![An example rubric in CodeGrade with 2 manual and 4 automatic rubric categories.](/files/ttI8ET5c43Bw8vqzTAWD) {% hint style="success" %} Don't forget to press the "Submit" button once you're done configuring your Rubric! {% endhint %} # Add Automatic Grading Let CodeGrade AutoTest do the dirty work for you and automate (a part of) your grading! The last step of setting up your assignment is to set up automatic tests. To create an AutoTest, navigate to the AutoTest tab, select AutoTest v2, then press Create. Alternatively, use the Copy option to copy an AutoTest setup from another assignment you have created. Next, click on any of the languages below to get a language-specific guide for setting up autograding. {% content-ref url="/pages/jj72ts0v49Yks7MZ2EGa" %} [Create your first Python assignment](/automatic-grading-guides/python/create-your-first-python-assignment) {% endcontent-ref %} {% content-ref url="/pages/3SWN5XkkYLx5GarvlxWh" %} [Create your first Java assignment](/automatic-grading-guides/java/create-your-first-java-assignment) {% endcontent-ref %} {% content-ref url="/pages/iSdvQe08LPiIovScq06T" %} [Create your first C assignment](/automatic-grading-guides/c/create-your-first-c-assignment) {% endcontent-ref %} {% content-ref url="/pages/KbSE5Rx8a1S22kb9Nepu" %} [Create your first C++ assignment](/automatic-grading-guides/c++/create-your-first-c++-assignment) {% endcontent-ref %} {% content-ref url="/pages/mMtdUF4amK8wlACxi2nV" %} [Create your first C# assignment](/automatic-grading-guides/c-1/create-your-first-c-assignment) {% endcontent-ref %} {% hint style="success" %} Make sure to check the list of Automatic grading guides for more languages! {% endhint %} {% hint style="success" %} Is your language not listed? Get in touch with us at for a specialized consultation! {% endhint %} # Student View Experience assignments as students would When developing your assignment, it can be useful to see how it looks and works from the student's perspective. For example, you may want to check that certain information is not visible to them or that their workflow for submitting and receiving feedback is as desired. This is exactly what the **Student View Mode** is meant for. **NOTE:** Student View is available to users with specific permissions. To enable access to Student View, a user must have a role that has the **"View course roles"** permission. Teachers and administrators can manage the permissions assigned to each role within their courses. To learn more, visit [Managing Course Permissions](https://help.codegrade.com/setup-assignment/other-features/configuring-permissions-of-roles). To turn it on, go to an assignment and then click the :eyeglasses: icon in the bottom left corner of your CodeGrade interface.

Activating the Student View in CodeGrade.

This view grants permissions typical of the Student role, with the added possibility of seeing hidden assignments. The account's original identity is retained. This means that there is no relation with the Test Submission. A submission created in Student View will still refer to your original account but with student permissions. Once activated, you will be brought to the submission page for the current assignment, where you can submit to see the feedback a student would receive.

Student View activated for an assignment. Arrows indicate how to leave the view.

\ To leave the Student View, you can click the :eyeglasses: icon or the "Leave Student View" button that will be present at the bottom of the page. # Grade your assignment Now that you have builded your assignment and your students have handed in, it's time to grade it! If you are fully automatically grading your assignments, you don't have to do any grading! But, you might want to give your students some personalized feedback. {% content-ref url="/pages/-MMKmqfIQu40QVNbqYEk" %} [Give Feedback](/setup-assignment/grade-your-assignment/grading-and-giving-feedback) {% endcontent-ref %} If you are (partly) manually grading your assignment, you also have to fill in the rubric or assign a grade. Moreover, you can also override the grade if it was automatically given. {% content-ref url="/pages/FeTdE94yW6RXlMw9KQZZ" %} [Give a Grade](/setup-assignment/grade-your-assignment/give-a-grade) {% endcontent-ref %} # Give Feedback Once your students have handed in their work, you can use CodeGrade's feedback features to give your students detailed and context-specific feedback on their code. CodeGrade offers two options for giving students feedback: * **Inline:** Click on a line of code to provide context-specific feedback. * **General:** Give feedback about students' general performance and points of improvement. {% hint style="success" %} Markdown is supported in all feedback features. {% endhint %} ## Inline Feedback
1. Navigate to the **Submissions** page and click on one of your students' submissions. 2. If the student has submitted multiple files, use the **Code** panel on the right of the screen to navigate to the right file. 3. Click on a line you want to give feedback on. This will open the **Feedback Panel** which you can use to write your comments. 4. Click **Save** or press *Ctrl + Enter* to save your comment. ## General Feedback ![Giving general feedback on an assignment](/files/2JRstdz60yCNX7fTgofI) 1. In your student's submission, navigate to the **Feedback** tab. 2. Under **General Feedback**, click to start a new thread and type your feedback. ## Creating snippets With snippets you can easily autocomplete commonly used feedback. CodeGrade offers both personal and course-wide snippets. Personal snippets can be managed in the Profile Panel in the sidebar. Course-wide snippets can be managed in the Course Settings Page. ![](/files/4LqpgT1DRMHUOMjXry5m) 1. To create a Snippet, open the **Feedback Panel** by clicking on a line of code or editing an existing comment using the **Edit** button (pencil icon). 2. Press the **Snippet** button (scissors icon) to open the **Snippets Panel**. 3. Enter a keyword for the snippet and confirm. 4. Now, you can type your keyword into any Feedback Panel and select the Snippet to autofill your comment by clicking on it or pressing *Tab* and then *Enter* on your keyboard. ## Replying in a thread Feedback in CodeGrade is not just one-way. Each new comment an instructor makes in a submission is a new **Thread**. Using the **Reply** feature you and your students can have a discussion in each thread. 1. In your student's submission, click the Reply button (back arrow icon) to reply to a comment. 2. Type your message and click Save. {% hint style="success" %} Grades and feedback are only visible to students after toggling the "Grades Published" toggle on the Assignment Management Page's general tab. Toggling this will also automatically synchronize your grades with your LMS. {% endhint %} # Give a Grade Now that you have given your students feedback, you can award a grade to their submission to keep track of their academic progress. Ultimately, the **Final Grade** is what counts. But, CodeGrade offers two mechanisms for awarding grades: * You can directly set the Final Grade manually. * Or, you can use Rubrics to award points that count towards the Final Grade. If you have set up a Rubric for your assignment, then the total number of points across all Rubric Categories is considered the maximum number of points for the assignment. This translates to a Final Grade of 100% or the maximum amount in whatever points scale you have chosen. ## Manual Rubric Categories Rubric Categories connected to automatic tests will be filled in automatically. But, those that aren't connected to automatic tests must be filled in by you manually. Ideally, you should use Discrete Rubric Categories for manual grading. 1. Navigate to your student's submission. 2. Select the manual Rubric Category that you want to grade. (Be aware that CodeGrade automatically collapses all Rubric Categories into a drop-down menu if your screen is small.) 3. Select one of the available points steps. ## Final Grade Once all of the Rubric Categories have been filled in, the number of points awarded to the student will count as their final grade. This is displayed at the bottom of the screen according to your chosen grading scale. You can manually override the final grade by clicking on it, entering a new value, and submitting. To revert to the original rubric grade, click the circular arrow icon labeled *Reset to the rubric grade* in the grading bar. ![Opening the rubric and inputting your final grade](/files/fIcTeuvkrE5eVzfV7R3z) {% hint style="success" %} Grades and feedback are only visible to students after toggling the "Grades Published" toggle on the Assignment Management Page's general tab. Toggling this will also automatically synchronize your grades with your LMS. {% endhint %} # Analyze your assignment Finally, you can analyze the performance of your assignment and check if students potentially committed plagiarism. {% content-ref url="/pages/-MMKlbVFRXtgnmFVbMee" %} [Detect Plagiarism](/setup-assignment/analyze-your-assignment/checking-for-plagiarism) {% endcontent-ref %} {% content-ref url="/pages/-MMKmkKRDOxpJM\_8Rzf7" %} [Analytics](/setup-assignment/analyze-your-assignment/viewing-analytics) {% endcontent-ref %} You can find overviews of all grades per assignment and per course, you can also export grades for your own administration. [Grades are synchronized with your LMS when the assignment state is Done.](/faq/releasing-grades-and-feedback) {% content-ref url="/pages/tmNsPYPE7e3WHBDPCGEI" %} [View and export grades](/setup-assignment/analyze-your-assignment/view-and-export-grades) {% endcontent-ref %} If you want to run your own analytics, you can easily export data from CodeGrade from within the assignment or with our API. {% content-ref url="/pages/PLlbrko8meyhqNqEkXvt" %} [Exporting Submissions](/faq/exporting-submissions) {% endcontent-ref %} # Detect Plagiarism CodeGrade provides a built-in plagiarism detector to find possible plagiarism cases in CodeGrade assignments. The plagiarism detector compares each student's submission with each other and flags incidents where the students' code is identical. The detector can make this comparison robustly even when students have attempted to hide the plagiarism by placing the code structures in a different order or renaming their variables. ![Starting a plagiarism run on an assignment](/files/LktT0oCCWMhivKVhR3cz) ## Run Plagiarism Detector 1. Go to the **Assignment Management** page by clicking the Settings button (gear icon) in your assignment. 2. navigate to the **Plagiarism** tab. 3. Set up the options to apply to your assignment: * **Language**: Language of the code in the assignment. * **Suffixes to include:** Comma separated list of suffixes of files to include in the check. Leave empty to use the default suffixes for the selected language. * **Minimal similarity:** The minimal average similarity to include in the results, the default is an average of 50%. * **Old assignments:** Optionally, select an old assignment to include the submissions in the plagiarism check. * **Old submissions:** Optionally, upload an archive of submissions to be included in the plagiarism check. * **Base code:** Optionally, upload an archive with code to be excluded from the plagiarism check. 4. Click the **Run** button to start the plagiarism check. A plagiarism run will be displayed at the top of the screen. ## Review Plagiarism Cases 1. Click on the plagiarism run to see an overview of detected plagiarism cases. 2. The **Plagiarism Overview** page displays all cases where more than the threshold percentage of code was identical. Click on any report to investigate the specific case. 3. The **Plagiarism Report** shows the submissions suspected of plagiarism side-by-side. The identical sections of code are highlighted in the same color according to the legend. Clicking on one of the items in the legend will automatically display the sections of code side-by-side. {% hint style="success" %} Click the **Export** button on the top right hand corner to export a plagiarism report. Plagiarism cases can be exported in Docx or LaTeX format, so that additions and further comments can be added before exporting as PDF. {% endhint %} # Analytics ![Navigating to the Analytics of an Assignment](/files/wx27d85tLjgcfBNNVrYk) CodeGrade offers a full, customizable analytics dashboard: 1. In your assignment, Navigate to the **Analytics** tab. 2. On the Analytics dashboard under the General Statistics, you'll find the **Filters** option which you can use to generate detailed graphs. 3. As an example, you may want to group each statistic by grader. To do that, click on the **Split** button (scissors icon) and select the graders you want to group by from the dropdown list. Down below, you'll see all the graphs generated accordingly. 4. Customize each graph using the settings in the top right corner. {% hint style="success" %} Filters and settings are not saved in the app but in the URL, making sharing easy. Copy and paste the URL to save or share your presets. {% endhint %} # View and export grades You can find an overview of all grades per assignment or per course (in the gradebook). You can also export grades in both of these places. ## Export grades for an assignment You can easily export all submissions and all grade and feedback data of an assignment. You can also filter the export using search or by assigned user.

Exporting grades and feedback for an individual assignment.

1. Navigate to the **Assignment Home** Page of the assignment you want to view or export grades for. 2. You will find a list of all grades on the **"Submissions" tab**. 3. Navigate to the **"Export" tab**. 4. Click "**Export metadata**" to export the grade and feedback data of the current assignment to a CSV. ## Export grades for a course You can also find an overview of grades across all assignments in a course in the **Gradebook**.

Viewing the gradebook for a course in CodeGrade and downloading a CSV with all grades.

1. Navigate to the **Course Settings Page** of the course you want to find or export all grades for. 2. Navigate to the **"Gradebook" tab**. 3. You will find a list of all students, grades, and assignments on this page. Click **"Download CSV"** to download a CSV with all grades. # Other features CodeGrade offers many other features, if you are interested in using those, check them out below. {% content-ref url="/pages/ET4yxvKoM1sdYG9pNJ4U" %} [Copy-Paste Limit in Editor](/setup-assignment/other-features/copy-paste-limit-in-editor) {% endcontent-ref %} {% content-ref url="/pages/Lmw4GjNaLKnwT0lcJTvB" %} [Using the Terminal](/setup-assignment/other-features/using-the-terminal) {% endcontent-ref %} {% content-ref url="/pages/Ertcu6PaPcwSfk6qbS3H" %} [Running Secure Exams](/setup-assignment/other-features/running-secure-exams) {% endcontent-ref %} {% content-ref url="/pages/YQvFyzZirZyGMJyCn4ul" %} [IP Restriction](/setup-assignment/other-features/ip-restriction) {% endcontent-ref %} {% content-ref url="/pages/uooGHuAeUHxpEV17gMnw" %} [AI Assistant](/setup-assignment/other-features/ai-assistant) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIaLtrLi8WydKWVb" %} [Peer Feedback](/setup-assignment/other-features/setting-up-peer-feedback) {% endcontent-ref %} {% content-ref url="/pages/-MMKlImjUnQ5KPfla\_7S" %} [Group Assignments](/setup-assignment/other-features/making-a-group-assignment) {% endcontent-ref %} {% content-ref url="/pages/-MMKlkccFKT2aJpsQ4r8" %} [Collaborative Grading](/setup-assignment/other-features/dividing-submissions-over-graders) {% endcontent-ref %} {% content-ref url="/pages/-MdX453UycGQL7OiUwem" %} [Anonymous Grading](/setup-assignment/other-features/grading-anonymously) {% endcontent-ref %} {% content-ref url="/pages/-MNZ-wkfulkIe1PlAxjo" %} [Manage Course Permissions](/setup-assignment/other-features/configuring-permissions-of-roles) {% endcontent-ref %} {% content-ref url="/pages/-MMKlIkVJ8t9Gw5VpOUI" %} [Hand In Requirements](/setup-assignment/other-features/setting-up-hand-in-requirements) {% endcontent-ref %} # Copy-Paste Limit in Editor When students write code in CodeGrade's in-browser editor, you can now set an upper limit on how much text they can paste at once. This discourages students from pasting in code they didn't write, for example from AI tools or from other students.
The "Allow copy-pasting" slider in the Submission settings

The "Allow copy-pasting" slider in the Submission settings

#### How the paste limit works * By default, copy-pasting is **Unlimited** and students can paste freely. * Moving the slider towards **Strictest** lowers the maximum number of characters students can paste at once in the editor. * At the strictest setting, pasting is blocked completely. * The limit only applies to students. Staff can always copy and paste in the editor and preview the restriction in Student View. {% hint style="warning" %} This feature deters students from copying and pasting, but does not entirely prevent it. Students can still type code over from another screen. For high-stakes assessments, combine it with exam features covered in [Running Secure Exams](/setup-assignment/other-features/running-secure-exams). {% endhint %} #### Setting a paste limit 1. Go to the **Assignment Settings** page of your assignment. 2. Under the **General** tab, locate the **Submission settings** section. 3. Make sure the **Editor** submit method is enabled. The paste limit only applies to the in-browser editor, not to the File uploader or Git submissions. 4. Drag the **Allow copy-pasting** slider to the limit you want, from **Unlimited** to **Strictest**. The selected limit (e.g. "Up to 30 characters") is shown above the slider. 5. Click **Save** to apply your changes. {% hint style="info" %} Use [Student View](/setup-assignment/build-assignment/student-view) to preview what your students will see when they try to paste more than the limit. {% endhint %} #### What students see When a student tries to paste text that exceeds the limit, the paste is blocked and a message explains what they can do instead:
Pasting more than the character limit

Pasting more than the character limit

Pasting turned off entirely

If pasting is blocked completely, students are asked to type their code in instead. # Using the Terminal {% hint style="info" %} This feature is still in Beta and can be enabled on request. Contact us at if you are interested in trying it out! {% endhint %} Students can now run code interactively, and learn to use the Ubuntu terminal in CodeGrade. As a teacher, you control whether the Terminal is available for an assignment, and what it can access. ## Enabling the Terminal 1. You can enable the Terminal in an assignment’s **General Settings** by clicking on the toggle in the top-right corner of the **Terminal** panel.

Terminal panel in the General settings.

2. You’ll see the following options: * **Allow internet access**: students can browse the web from the Terminal. This is on by default. * **Allow access to uploaded files in AutoTest**: students can access files uploaded in the ATv2 configuration in the fixtures folder. This is off by default. A Terminal is specific to a snapshot. If you'd like to change a Terminal's dependencies in the AutoTest configuration, you will have to build and publish a new snapshot. You can choose the Ubuntu version as well as install dependencies by clicking on “Configure Terminal in AutoTest”. Otherwise default settings will be used, i.e. the latest version of Ubuntu supported by CodeGrade (currently Ubuntu version 26.04). ### Working with files Files from the Code Editor are mirrored into the Terminal at `~/student`. This directory is read-only: students can't edit or delete these files from the Terminal itself; any changes still need to be made in the Code Editor. Compiled output and any new files students create must be written somewhere outside that directory. Nothing students write in the Terminal persists once the session ends, and none of it affects the files in the Code Editor. Files uploaded in an AutoTest configuration are placed in `~/fixtures`. ## Where students find the Terminal In the Code Editor, students can access the Terminal by clicking on the icon in the bottom left of the screen.

Terminal icon in the Code Editor.

Open Terminal in the Code Editor.

The Terminal session is discarded with the editor session, i.e. nothing in the Terminal is saved once the student hands in their work. Closing the tab will reset the Terminal as well. The Terminal also keeps whichever configuration was active when a student started working, for that entire session. If you change the Terminal settings afterward, students already working aren’t affected until they start a new session. Students can open up to 5 Terminal tabs by clicking on the "**+"** at the top of the Terminal window. Each tab maintains its own independent terminal session. Tabs are restored when the editor is reloaded, so work isn't lost.

Multiple Terminal tabs open.

{% hint style="info" %} Terminal settings are copied automatically when you import an assignment, so you don’t need to reconfigure them. {% endhint %} # Running Secure Exams Ensuring a secure and fair exam environment is essential. CodeGrade allows you to conduct exams with confidence by offering robust access controls. By setting up a dedicated exam course, you can manage all restrictions at the course level, ensuring every exam runs smoothly and securely.

Student view of an active exam session with Session Lockdown enabled

## Step 1: Set Up an Exam Course Create a dedicated course specifically for exams. This provides a centralized place for all exam access controls. ## Step 2: Manage Course‑Level Access Restrictions Navigate to ***Course Settings → Access*** {% hint style="warning" %} **Missing the Access tab?** * **Course Settings → Access** requires the **Edit access restrictions** permission. * **Assignment Settings → Access** requires the **Edit assignment access restrictions** permission. Permissions can be managed in **Course Settings → Permissions**. {% endhint %} Here, you can configure: * **Password Protection**: A course-level password that students must enter to access the course. * **IP Restriction**: Limit access to certain IPv4 addresses or ranges. * **Session Lockdown**: When enabled, students are restricted to this course and cannot access other courses or platform features such as search or snippets. Any attempt to do so will result in token or access-related errors. * **Idle Session Detection:** *(Enabled by default when Session Lockdown is turned on.)* Ends the exam session after 5 minutes of inactivity, even if the student's laptop stays unlocked. Each interaction extends the session by another 5 minutes, up to the cap set at entry. The setting is remembered if you toggle Session Lockdown off and on. Changing this setting mid-exam will not affect students who have already entered the exam. * **Entry Limits**: Set how many times a student can enter the course (e.g., one entry). You can adjust per-student overrides in the entry log. * **Entry Window**: Set the start and end date and time between which students are allowed to enter the exam. Entry attempts before the start time or after the end time are rejected. Changing the start time mid-exam will not affect students who have already entered. Changing the end time mid-exam will only affect students who have already entered if Idle Session Detection is also enabled for the exam. Save your changes to apply them to the entire course.

Course Settings → Access, showing Session Lockdown enabled and Idle Session Detection turned on for an exam course.

**How the idle session detection works**\ \ Exam sessions stay alive automatically while students are active on the exam tab, and send an immediate heartbeat when a student returns from a locked laptop or a backgrounded tab. If the heartbeat fails, the student sees a clear session interrupted message and is asked to log back in. With Idle Session Detection enabled, 5 minutes of inactivity ends the session even if the device is still unlocked. ## Step 3: Monitor and Adjust Entries In the same ***Access*** section, you’ll find the entry log. It shows: * Each student’s name and username * The number of successful entries * The limit set for them You can override or reset limits if needed. ## Step 4: Assignment-Level Access (Optional) If necessary, you can also set access restrictions for individual assignments. Navigate to ***Assignment Settings → Access*** Here, you can add assignment-specific password or IP restrictions if required. ## Best Practices * Use IP restrictions for on-site exams (e.g., campus networks). * Share passwords securely, just before the exam. * Use "Student View" to test the entire setup, including Session Lockdown, restrictions, and Heartbeat Enforcement if enabled. * If Session Detection is turned on, let students know that stepping away from the exam for more than 5 minutes will end their session and require them to log back in. * If both course-level and assignment-level restrictions are applied, course-level rules will apply broadly while assignment rules add further control. * If a change to the entry window end time doesn't seem to affect students already taking the exam, check that Idle Session Detection is enabled — end time changes only apply to active sessions when it is on. For students, once they start the exam course, their session is locked. Any attempt to switch courses or reuse a session will lead to access errors, ensuring a secure, focused exam environment. # IP Restriction IP restrictions let you control which students can access an assignment based on their network location. By configuring IPv4 addresses or CIDR ranges, only students connecting from those addresses will be able to open and work on the assignment.

Configure IP restriction in assignment settings

### How IP Restriction Works * By default, all IP addresses are allowed to access an assignment. * Once one or more restrictions are added, only the specified IPv4 addresses or ranges are permitted. * At present, IPv6 addresses are not supported. {% hint style="info" %} Students connecting from an address outside the allowed range will be blocked from opening the assignment. {% endhint %} *** ### Adding IP Restrictions **Access the Settings**\ Navigate to *Assignment Settings* > *Access*. **Add a Rule** 1. Under *IP Restriction*, enter either: * A single IPv4 address (e.g., `1.2.3.4`) * A CIDR range (e.g., `1.0.0.0/24`) 2. Save your changes by clicking *+ Add Rule* or hitting *CMD+Enter*. {% hint style="success" %} Use [**Student View**](/setup-assignment/build-assignment/student-view) to preview and confirm that only the allowed IP addresses can access the assignment. {% endhint %} *** #### Examples * `1.2.3.4` → Only this single IP can access the assignment. * `1.0.0.0/24` → Allows all IPs from `1.0.0.0` through `1.0.0.255`. * `1.1.1.0/24` → Allows all IPs from `1.1.1.0` through `1.1.1.255`. {% hint style="warning" %} Misconfigured rules can prevent valid students from accessing, so always test with your own IP address before applying strict restrictions. {% endhint %} # AI Assistant CodeGrade allows you to configure multiple AI assistants tailored to different educational purposes for an assignment. Here’s an overview of the available assistants and how to configure them: {% hint style="info" %} The AI Assistant is currently in Beta. Please email for more information on joining the Beta and getting access. {% endhint %} *** ### Types of AI Assistants **Learning Assistant** * Designed to focus on learning and understanding. * It does not provide direct coding answers but helps students understand principles, concepts, and feedback from AutoTest. **Feedback Assistant** * Can provide detailed feedback on code from a code quality perspective. **Coding Assistant** * A powerful tool that assists students with coding by generating code and providing solutions. This gives students unparallelled access to any coding questions. *** ### Student Access to AI Assistants * As an instructor, you can enable access to one or multiple AI assistants for students. Students can start a new chat with any enabled assistant configuration. * The AI assistant is integrated into the **Editor**, allowing students to use it seamlessly while coding. * Within the Editor students can click on the Assistant icon in the right-side bar or in the top-right corner of the screen:

AI Assistant icons in the Editor.

* If you’re using **Git submission** or **File Uploader**, students can access the assistant through the **Submission** overview. * Both students and instructors can view and review all conversations with the assistant. *** ### Viewing Assistant Chats * When the AI assistant is enabled, a dedicated **Assistant Chat** tab appears in the Submission Overview.

Submission Overview: Assistant Chat Tab

* This tab is accessible to both students and instructors, displaying all chat interactions. *** ### Configuring AI Assistants **Access the Settings** * Navigate to **Assignment Settings > Assistant**
**Create assistant** * Click on "Create new assistant" and choose the type of assistant you'd like to add.
**Select a Model for the Assistant** * After creating an assistant, choose which AI model it will use to generate answers. **Customize the Assistant** * Configure **Content Guardrails** to control what the assistant is allowed to respond to. * Enable **Content Access** to let the assistant read the student’s current code, files, quiz, notebook, AutoTest results, and assignment description during the session. It only sees content available to the student. When disabled, the assistant responds using the conversation alone. * Edit the **System Prompt** to customize the assistant’s behavior, tone, and instructions.

Configuring a Learning Assistant with Context Access

* Adjust settings such as the programming language or specific topics the assistant should address. **Set the Status** * Set the assistant to **enabled** to allow students to start using it. * Switch to **disabled** to stop new interactions while retaining access to previous chats for review. *** ### Key Notes * Once the status of an assistant is disabled, all prior chats remain accessible to both students and instructors. * Configuring and customizing AI assistants provides flexibility in aligning them with your teaching objectives. With these tools, CodeGrade makes it easy to enhance learning, provide valuable feedback, and assist students with coding in an integrated, efficient way. # Peer Feedback {% embed url="" %} Peer feedback is a powerful tool that provides students the opportunity to learn from each other and improve their own skills. With peer feedback, students will be able to review and [give inline comments](/for-students/advanced-features/giving-peer-feedback) and general feedback to their peers. Teachers can approve or disapprove these comments and optionally grade the given feedback. Follow the steps below to learn how to set this up for your CodeGrade assignment. 1. Go to the **Assignment Settings** page of the assignment you want to turn on peer feedback for by clicking on the **Settings** button. 2. Under the **General** tab, find the **Peer Feedback** section and click '*Set up Peer feedback*'. 3. Thereafter, you'll see the following options:
* **Amount of students:** Enter the number of students that each student has to give feedback to. (i.e. one student has to review $$x$$ other students) * **Time to give peer feedback:** Define the number of days and hours that students have for providing peer feedback after the passing of original deadline of the assignment. * **Automatically approve comments:** If switched to *yes*, all peer feedback comments by students are automatically approved. If switched to *no*, the teacher has to manually approve or disapprove all comments.
4. When the values are set up according to your requirements, click on the **Submit** button to turn on the peer feedback feature. Clicking on **Disable** will turn the feature off. ![Peer feedback basic setup for an assignment](/files/L9dUCQshGDXcQyK4BUOz) When Peer Feedback is turned on, a random division is automatically made after the deadline of your assignment and students can start giving peer feedback. In case you want to view comments provided by students after the deadline for the peer feedback has passed, go to the **Submissions** page and click on any submission to find comments under the **Peer Feedback** tab. As a teacher, you have the option to approve or disapprove the comments by students. It is also possible to reply to or edit any comment. Students will be able to view their Peer Feedback only [after you release grades and feedback to them](/faq/releasing-grades-and-feedback). #### Allow your students to see AutoTest results of peers during Peer Feedback Do you give your students instant feedback using AutoTest? Or automatically generate files you want students to see during Peer Feedback? You can allow your students to see AutoTest during giving Peer Feedback by [giving them the *"View peer reviewed user's AutoTest details"* permission](/setup-assignment/other-features/configuring-permissions-of-roles). {% hint style="success" %} Add a rubric category to manually assess and give a grade to peer feedback. As all peer feedback given by a student can be found on the student submission page, you can easily grade their peer feedback. {% endhint %} {% hint style="success" %} As a teacher, you will always be able to manually disapprove student inline comments, even if **automatically approve comments** is turned on. {% endhint %} {% hint style="info" %} Peer Feedback will allow your students to give inline comments and general feedback to each other, rubrics and grades are still filled in by the teacher. {% endhint %} # Group Assignments Allow your students to hand in as groups by making your CodeGrade assignment a group assignment. Groups for group assignments are managed entirely within CodeGrade. Group sets are created in CodeGrade on a course-level, meaning that multiple assignments within one course can have the same groups. {% hint style="info" %} It is important that a regular individual assignment is created within your LMS, and **not** a group assignment. CodeGrade handles all the groups and makes sure grades and feedback are passed back correctly to the corresponding group members. {% endhint %} ![Adding a group set to your course in the Course Manager](/files/sE6dvKeO16vwdw10mZ2t) 1. A group set for your course in CodeGrade has to be created first. Click the **Settings** button next to your course to go to the **"Course Settings"** page. 2. Navigate to the **"Groups"** tab to find an overview of previously created group sets or to create a new group set. 3. Click the **"Add group set"** button to create a new group set, specify the minimum and maximum group sizes and press **Save**. 4. Now that a new group set is created, this group set can be linked to assignments. Go to the **"Assignment Settings"** page of the assignment you want to make a group assignment by clicking the **Settings** button. 5. On the **"General"** tab, scroll down to the **"Group Assignment"** section. Here, select the desired group set and confirm by pressing **"Submit"**. Your assignment is now a group assignment! {% hint style="success" %} Set your minimum group size to **1**, so that individual users that are not in a group can still hand in submissions too. {% endhint %} ![Connecting the created group set for an assignment in the Assignment Manager](/files/KQH6KXn5gdC9iEWdneMm) By default, students can create new groups and join groups within a group set themselves. The names of groups are automatically generated but can be manually changed afterwards. If you want to change this default behaviour (e.g. make it impossible for students to create or join groups themselves). {% hint style="success" %} Want to exclude individual students from working in a group? Create a new role called "*Individual Student*" and give that role the "*Upload without group*", now all students with this role can submit their work without enrolling in a group! Read more about [setting up permissions and roles here](/faq/configuring-roles-of-members#creating-a-new-role). {% endhint %} # Collaborative Grading {% embed url="" %} CodeGrade offers multiple tools to divide submissions over the graders in your course. Graders can be divided manually or randomly. Additionally, divisions (both manual or random) can be reused over multiple assignments. Dividing submissions over your graders has multiple advantages. In addition to having a clear way to communicate with your graders, it also allows your graders to easily filter the submission lists on the CodeGrade website and in the Filesystem to only show their submissions to grade. ### Manually dividing submissions ![Manually selecting graders on the Submissions list](/files/kr7mKxCmYKhGbQqIOe6H) 1. Navigate to the **"Submission list"** of an assignment, by simply clicking on the name of the assignment. 2. If available, uncheck the **"Assigned to me"** filter to display a list of all submissions. 3. It is now possible to manually divide the submissions by using the dropdown in the **"Assigned to"** column. This dropdown shows all graders available in this course. 4. Click any grader in this list to assign the submission to this grader. ### Randomly dividing submissions ![Randomly dividing all submissions among graders by percentage / weight in the Assignment Manager ](/files/upNYz5TOSCYKvIR5X2Mn) 1. Go to the **Assignment Settings** page for the assignment. 2. Navigate to the **"Graders"** tab to find the grading dashboard with a list of all graders available in this assignment. 3. Select the graders you want to divide the submissions over in the **"Divide Submissions"** section, optionally adjust the weight or percentages of the graders. 4. Press the **"Divide"** button to the divide the submissions over the selected graders in respect to the given weights. ### Connecting divisions between assignments ![Connecting a previous division to the current assignment in the Assignment Manager](/files/xGoNfx4KcPByEevqmJeF) It is possible to reuse both manual and random divisions in other assignments. For instance if you require the same graders to grade the same students / submissions for all assignments. Once you have divided the submissions on one assignment following either of the above methods, you can follow the steps below to connect this division to other assignments too: 1. Go to the **Assignment Settings** page. 2. Navigate to the **"Graders"** tab to find the grading dashboard with a list of all graders available in this assignment. 3. In the **"Divide Submissions"** section, use the *Connect divisions to* dropdown to select the assignment you want to connect the divisions to. {% hint style="warning" %} Changing divisions in an assignment that is connected to a previously divided assignment will not change the division for the previous assignment **or** for other assignments connected to this previous assignment. {% endhint %} # Anonymous Grading Sometimes you may prefer to grade anonymously yourself or have your graders grade anonymously. CodeGrade assignments have a setting to hide all occurrences of student and group names in our user interface. This guide explains how you can turn this on for your assignment. ![Turning on anonymous grading for your CodeGrade assignment.](/files/pZPZk5h94af7tEiE5wGO) 1. Navigate to the **Assignment Settings page** of the assignment you want to start grading anonymously. 2. Go to the **General** tab and navigate to the **Anonymized grading** option. 3. Toggle to **Yes** to turn on and toggle to **No** to turn off. Press **Submit** to submit your changes. ![List of students with Anonymous grading turned on. ](/files/s1Yz7t0Im7gjQ92YG41I) {% hint style="warning" %} Anonymous grading is a UI-based feature, it hides all occurrences of student and group names in our user interface. Names in code, the Filesystem folders, exports and plagiarism cases are **not** anonymized. You may disable some of these functions for your graders [using our permission system](/setup-assignment/other-features/configuring-permissions-of-roles). {% endhint %} {% hint style="info" %} Users with the [permission](/setup-assignment/other-features/configuring-permissions-of-roles) "*See anonymized names*" can see student and group names even if Anonymized grading is turned on. {% endhint %} # Manage Course Permissions {% embed url="" %} You can change permissions of default roles and [newly created roles](/faq/configuring-roles-of-members). Almost all features and actions in CodeGrade have a corresponding permission, with which you can enable or disable them for certain members in your course. ![Managing course permissions in the Course Manager](/files/LVR7PyQQzVUZR6tnigDX) 1. Navigate to the **Course Management** page of your course and go to the *Permissions* tab. 2. Search for the permission you want to enable or disable. 3. Enable this permission for the role you want by checking the box in column of that role. 4. Changes take effect immediately. {% hint style="success" %} Users may need to reload their page for permissions to take effect. {% endhint %} #### Common practices: Permissions allow for endless possibilities. Some common practices are: * Allowing students to hand in after the deadline. {% content-ref url="/pages/-MN-A7QLTJ2k44qHcAT8" %} [Allowing students to hand in after the deadline](/faq/allow-students-to-hand-in-after-deadline) {% endcontent-ref %} * Allowing students to see inline comments or answers to their questions before the [assignment's grades are published](/faq/releasing-grades-and-feedback). Give your students the **See feedback before grades published** permission. * Let Teaching Assistants help you set up parts of the assignment. For instance helping you with configuring AutoTest (give TAs the **Edit the configuration of an AutoTest** permission) or with your rubric (give TAs the **Manage rubrics** permission). {% hint style="danger" %} Some permissions allow users to do irreversible actions (delete submissions) or change their own permissions. Change permissions highlighted in red with caution. {% endhint %} # Hand In Requirements {% embed url="" %} It is desirable to have strict hand-in requirements and instructions for some programming assignments. Perhaps to make manual or automatic testing less cumbersome or to disallow the handing in of certain unwanted files. CodeGrade offers hand-in requirements for assignments that *force* students to hand in files or archives with a requested structure. Follow the guide below to learn how to set this up for your CodeGrade assignment: ![Setting up Hand In Requirements in the Assignment Manager](/files/NtVpcAIIKoX3lcwXNxDu) 1. Go to the **Assignment Settings** page for the assignment you want to add hand-in requirements. 2. Under the **General** tab, locate the **Hand in requirements** section. You can choose to copy the instructions over from another assignment, or set up a new requirements schema. 3. Select the policy of your requirement: * **By default** *Deny all files*: All files are denied by default, further rules (*required* or *allowed files*) determine exactly and strictly what students can or must hand in (i.e. this will allow you to only allow certain files and deny anything else). * **By default** *Allow all files*: All files are allowed by default, further rules (*required* or *denied files*) determine which specific files a student cannot hand in or must hand in (i.e. this will allow you to disallow certain files and allow anything else). 4. After selecting either of the policies above, more options are displayed to further specify the behaviour of your requirements: * **Delete empty directories**: If enabled, automatically delete empty directories in submissions. * **Delete leading directories**: If enabled, automatically delete superfluous leading directories (i.e. top-level directories in which all files / subdirectories are located). * **Allow overrides by students**: If enabled, the student can, after being shown a warning, still force hand in the submission even if it violates the hand-in requirements. Except for required files, those should always be present in the student submission. 5. Subsequently, rules to make exceptions for the set policy can be given in the **Exceptions and requirements** section. The following rules can be created: * **Required**: This rule is available to both default policies and can be used to require students to hand in certain files. * **Allowed**: This rule is available if all files are *denied* by default and can be used to make exceptions to this policy. Files and folders indicated as allowed are allowed to be handed in. * **Denied**: This rule is available if all files are *allowed* by default and can be used to make exceptions to this policy. Files and folders indicated as denied cannot be handed in. 6. When the default policy, options and exceptions and requirement rules are set up to your wishes, press the **Submit** button to save the hand-in requirements. {% hint style="success" %} Use `/` or `\` as a directory separator to specify that certain files are required, allowed or denied in a directory. Start the rule with a directory separator (`/` or `\`) to specify that a file is required, allowed or denied in the top level directory. To match more than one file, you can use a single wildcard for the name of the file, by using a `*`. For example `/src/*.py` matches any file ending with `.py` in the directory src that is directly in the top level directory of the submission. {% endhint %} ![Simple Hand In Requirement for a "Java Fibonacci Assignment"](/files/9omT6cIW6CNpvpbSfjAF) {% hint style="info" %} The exceptions and requirements are individual rules that act as exceptions to the chosen policy. There is no ordering between rules. {% endhint %} Once hand-in requirements are set up for an assignment in CodeGrade, students are provided with the **Hand-in instructions** view on the assignment upload page with an overview of the set up instructions and requirements. Furthermore, if a student submits a submission that does not conform the requirements, a warning will be shown. Unwanted files will automatically be detected and deleted. # AutoTest V2 Blocks This guide provides a comprehensive overview of all the blocks available in **CodeGrade's AutoTest Version 2**, which streamline the process of automatically grading assignments. The blocks are designed to handle a wide range of tasks, from executing test cases and validating code structure to managing outputs, controlling environments, and integrating grading rubrics. Each section outlines the function and configuration of different blocks, offering practical insights, best practices, and real-world examples to help you maximize the effectiveness of AutoTest in your grading workflows. Whether you are setting up a simple test or a complex assignment, this guide will walk you through the various options and how to leverage them to create a smooth and efficient grading process. *** ## 1. **Test Blocks** These blocks handle the actual execution of tests and grading based on program output, code structure or other types of tests.
IO Test The *IO (Input Output) Test* block is useful for validating program output against predefined expected results. Run a program and check if the output matches the expected output according to match blocks that are nested inside this block.

IO Test block running a compiled Java program and checking outputs for different inputs

There are three types of match blocks available for customizing the behavior: 1. **Full Match:** Verifies that the program's output exactly matches the expected output for a given input. 2. **Substring Match:** Checks whether the program's output contains the expected output as a substring for a given input. 3. **Regex Match:** Uses a Python regular expression to check if the output matches a specific pattern for a given input. {% hint style="success" %} All match blocks can be configured for **case sensitivity** and can either **ignore** or **include whitespace** during matching. {% endhint %}
Code Structure Test [Semgrep](https://semgrep.dev/) is a static analysis tool used to enforce coding patterns or syntax in student code. Run a *Code Structure Test* using Semgrep on a file to verify specific coding patterns in student submissions.

Code Structure Test block executing a Positive Match for a Java class definition pattern

**Steps for Setting Up a Code Structure Test with Semgrep** 1. **Specify the Student File:** In the `Student file` field, specify the file to check (default is every file in the student directory). 2. **Create Test Cases:** Nest Positive Match and/or Negative Match blocks inside this block to define your test cases: * **Positive Match:** Passes if a specified pattern is found in the student's code. * **Negative Match:** Fails if a specified pattern is found in the student's code. 3. **Craft the Rule in Semgrep Playground:** Visit the [Semgrep Playground](https://semgrep.dev/editor) to create the rule that defines the pattern to check for. 4. **Update the Rule Template:** Copy the output from the `Advanced` tab into the match block’s template and edit only the `pattern` and `languages` fields as shown below: 
rules:
     - id: untitled_rule
       pattern: YOUR_PATTERN
       message: Semgrep found a match
       languages: [THE_PROGRAMMING_LANGUAGE]
       severity: WARNING
{% hint style="info" %} For more information on using Semgrep, visit [Code Structure Tests with Semgrep](/automatic-grading-guides/code-structure-tests-with-semgrep). {% endhint %}
Simple Python Test The *Simple Python Test* block allows you to integrate student code into a larger Python script, providing enhanced grading and feedback capabilities beyond basic *I/O* testing.

Simple Python Test block to integrate student code into a complete program, defining necessary variables and checking the result

This block is especially useful for assignments where students are asked to submit small code snippets rather than full programs. **How It Works** You can insert the student's code into the script using the `# CG_INSERT filename.py` directive. This directive allows you to seamlessly integrate the student's code into a broader script where you can set up additional tests and validation steps. To further improve feedback, the `cg_feedback_helpers` module can be used. This module allows you to run the student's code in the context of a complete program and use assertions for more precise grading and feedback. To use `cg_feedback_helpers`, simply install it in the AutoTest setup with the following command:

Script block to install cg_feedback_helpers in AutoTest setup

{% hint style="info" %} For more information on using this block, visit [Advanced Python autograding](/automatic-grading-guides/python/advanced-python-autograding). {% endhint %}
Pytest [Pytest](https://docs.pytest.org/) is the industry-standard unit testing framework for Python. The *Pytest* block enables you to run Pytest unit tests automatically on student submissions. It is particularly useful for grading assignments where students develop their own functions, as it allows you to assess function correctness and edge cases.

Pytest block to test function definitions within the student submission

The *Pytest* block makes it easy to create multiple test cases to evaluate the functionality of different functions. This block is ideal for advanced assignments, especially those requiring full programs or handling edge cases. You can further enhance feedback with the `cg_pytest_reporter` module, which provides decorators like `@name`, `@description`, and `@weight` to add clarity and adjust the grading for each test case. For more information, visit the [cg-pytest-reporter documentation](https://pypi.org/project/cg-pytest-reporter/). Test results are clearly presented for easy review by students.

Pytest block test results are presented for easy review by students

{% hint style="info" %} For more information on using this block, visit [Advanced Python autograding](/automatic-grading-guides/python/advanced-python-autograding). {% endhint %}
Java Compile The *Java Compile* block is essential for compiling Java programs before any tests are executed. It ensures that the student's code is syntactically correct by handling compilation, displaying error traces, and providing detailed feedback if any errors occur.

Java compile block to compile all Java files in the student directory

Before running tests, every Java program must be compiled to detect syntax errors and other compilation issues. This block parses compiler errors and highlights the specific lines in the code where the issues occur, allowing for targeted feedback. By ensuring the program compiles successfully, it enables subsequent tests to focus on verifying functionality without the interference of compilation errors.
Junit5 The *Junit5* block allows you to define and run unit tests on compiled Java files in the student directory, providing a robust framework for evaluating assignments involving functions and classes.

JUnit5 block for testing the functionality of a compiled Java program

[JUnit5](https://junit.org/junit5/) is a widely used unit testing framework for Java that offers several advantages over basic I/O tests. It supports assertions, parameterized test cases, and advanced features like custom decorators (e.g., `@displayName`), which improve test readability. Additionally, JUnit5 enables weighted test cases, allowing for more granular grading. {% hint style="warning" %} Ensure the test class name and test file name matches. {% endhint %} This block can automatically install JUnit5, compile and execute the tests, or retrieve the necessary JUnit5 JAR files for custom configurations. The test results are then presented in a clear, user-friendly format, making it easy for both instructors and students to review and understand the outcomes.

JUnit5 block test results are presented for easy review by students

{% hint style="info" %} For more information on using this block, visit [Grading with JUnit5 and Checkstyle](/automatic-grading-guides/java/grading-with-junit5-and-checkstyle). {% endhint %}
Script The *Script* block allows the execution of a bash script as part of the test process. The block will pass if the script completes successfully, indicated by an exit status code of 0. If the script encounters an error or fails to execute as expected, resulting in a non-zero exit status code, the block will fail.

Script block executing a bash script to validate file I/O operation and exit with either a success or error code

The Script block is versatile and supports various tasks, such as: * File I/O validation * Environment setup and teardown * Multi-step integration tests * Checking external dependencies and process statuses
Custom Test The *Custom Test* block in CodeGrade’s AutoTest system allows for flexible and custom grading through scripting. You can write your own grading logic in any language (e.g., Python, Bash), making it ideal for complex assignments that don’t fit within predefined test blocks.

Custom Test block

**Key Features and Benefits of the Custom Test Block** **Flexibility in Grading:** Handle complex or multi-step evaluations, such as generating files, validating content, and performing detailed checks. Ideal for assignments requiring partial credit or advanced logic beyond predefined test blocks. **Custom Grading Logic**: Create tailored grading rules to assign partial points or grade based on custom criteria like output formatting or function efficiency. Results are returned in a simple JSON format (e.g., `{ "tag": "points", "points": "3/5" }`). **Integration with External Tools**: Run external libraries, tools, or custom configurations (e.g., linters, data analysis, or machine learning models) directly within your grading script, giving you full control over the testing environment. {% hint style="info" %} Explore an example of using the *Custom Test* block for C++ unit testing and code linting at [Advanced C++ autograding](/automatic-grading-guides/c++/advanced-c++-autograding). {% endhint %}
Flake8 The *Flake8* block runs the Flake8 linter on Python submissions to automatically generate inline comments based on the [PEP8 style guide](https://peps.python.org/pep-0008/).

Flake8 test on a Python program with default config

[Flake8](https://pypi.org/project/flake8/) is a widely-used linter that enforces coding best practices, particularly for beginners. The block requires only the file name and generates comments, highlighting lines based on severity (error, warning, info). Additionally, you can adjust the partial grade deduction for each severity level, allowing for more granular control over grading. You can also customize its configuration to include or exclude specific style rules.

Flake8 results are also displayed within the CodeGrade editor

The block generates comments directly on the student's code, highlighting the target lines according to the severity of the issue. Comments can be viewed by hovering over the line numbers, providing clear and actionable feedback. {% hint style="info" %} For more information on using this block, visit [Advanced Python autograding](/automatic-grading-guides/python/advanced-python-autograding). {% endhint %}
Checkstyle The *Checkstyle* block runs Checkstyle on selected files or folders, with the ability to use globs to define which files to include in the analysis.

Checkstyle test executing on a Java program

[Checkstyle](https://checkstyle.sourceforge.io/) is an industry-standard Java linter that enforces coding standards from various style guides. You can choose from three built-in style guides: Checkstyle's default template, Sun style guide, and Google style guide. Additionally, you can customize these guides to include or exclude specific rules. You also have the option to adjust the percentage of points deducted based on the severity of each comment (error, warning, info). Similar to [#pytest](#pytest "mention") block, this block also generate comments directly on student's code. {% hint style="info" %} For more information on using this block, visit [Grading with JUnit5 and Checkstyle](/automatic-grading-guides/java/grading-with-junit5-and-checkstyle). {% endhint %}
*** ## 2. Test **Configuration Blocks** These blocks help configure the testing environment or manage specific conditions for when tests should run.
Run Only In The *Run Only In* block ensures that the nested blocks are executed only within a specified environment.

Executing Java Compile test within every environment

Within CodeGrade, two distinct environments are available based on the submission settings: * **Submission-Only Environment**:\ This environment is activated when students submit files through uploads or Git repository connections. * **Editor Environment**:\ When the editor is enabled, this environment allows students to interact directly with their code. {% hint style="danger" %} Within editor environment, students can run tests multiple times using the *Run* button in the editor, with no limits on its usage. {% endhint %} {% hint style="success" %} Nest your test blocks within *Run Only In submission* block to require students to use a submission attempt to receive feedback. The *Run Only In* block can be applied to any test execution blocks. {% endhint %}
Run If The *Run If* block ensures that nested blocks are executed only when a specified condition is met.

Executes a compilation test if the specified rubric category score is 100% or higher

Executes a Script block if the specified rubric category score is below 25%

This block allows you to control the execution of test cases based on student performance. You can select a rubric category and define a threshold percentage, which dictates whether the tests inside the block will run. * If the student's score meets the specified threshold (either greater than or equal to or below the threshold), the tests will execute. * If the condition is not met, the tests will be skipped. This gives you flexible control over when and how tests are executed based on performance.
Allow Internet The *Allow Internet* block grants internet access to nested blocks.

Allow internet access for Script block execution

By default, internet access is enabled during the AutoTest Setup phase and disabled during the Tests phase. {% hint style="info" %} If a test execution block requires internet access (e.g., to install packages), wrap it within the *Allow Internet* block to enable connectivity. {% endhint %}
Timeout Each The *Timeout Each* block applies a timeout to all nested blocks, ensuring they complete within a specified duration.

Timeout execution of nested Junit5 test block after 15 seconds

The timeout must be a non-negative duration. By default, the timeout for a step is set to **120 seconds**. If you clear the input field, the step will revert to the default timeout, and any changes to the default timeout will automatically apply. This block is useful for preventing long runtimes and protecting virtual machine resources. By setting an appropriate timeout for each block, you can avoid issues like infinite loops or inefficient operations, such as poorly optimized recursive functions or loops. {% hint style="info" %} When combining I/O Test and Match steps, the timeout used for the match is the one set for the I/O Test step. {% endhint %}
*** ## 3. **Feedback and Result Management Blocks** These blocks manage how the feedback, results, and configurations are presented to the student.
Connect Rubric The *Connect Rubric* block links nested test blocks outcome to a specific rubric category, automating the grading process. You use this block to make your Test Blocks graded.

JUnit5 test results connected to the specified rubric category

Test blocks within the *Connect Rubric* block ensure that the outcome—whether pass, fail, or partial completion—is directly tied to the chosen rubric category. This integration allows the corresponding grade to be automatically recorded and passed back to the submission, streamlining the grading workflow. {% hint style="info" %} Results of test blocks executed outside *Connect Rubric* block will not be linked to the grade for submission. {% endhint %}
Weight The *Weight* block allows you to assign different weights to nested blocks, controlling their contribution to the overall score. This provides flexibility in adjusting the impact of each test on the final grade.

Use of Weight blocks to control impact of each test on final grade

{% hint style="info" %} By default, the weight of a test is set to **1**. Weights can be adjusted to be **negative**, **zero**, or **positive**, and the weights of nested blocks are multiplied together. {% endhint %} **Example Explanation:** In the image above, the *IO Test* block is connected to the rubric category **Result**. The total score for the **Result** category comes from two nested *Match* steps. * **Full Match Case #1** contributes **1 part** to the total score. * **Full Match Case #2** contributes **2 parts** to the total score. The weighted sum of these contributions determines the final score for the **Result** category, showcasing how different test cases can be given varying levels of importance within the rubric.
Hide The *Hide* block allows you to conceal specific elements from students. You can choose to always hide these elements or set them to be hidden based on the deadline or lock date.

Hide the configuration of the nested IO Test block always from the student

**Configurable Elements to Hide:** | **Config** | Settings of the block | | ---------- | ---------------------------------- | | **Output** | Output of the block | | **Result** | Whether the block passed or failed | **Behavior Across Different Test Blocks:** The impact of the *Hide* block varies depending on the type of test block. Below are some examples that demonstrate how different types of test blocks are affected by hiding:
Block ExampleWhat is Hidden
Hide config block around IO testHides the bash command, input, and expected output
Hide output block around IO testHides the actual output of the program
Hide config block around Code Structure TestHides the YAML configuration of the semgrep rule
Hide output block around Code Structure TestHides the output message about the pattern found or not found
Hide config block around Pytest/JUnit5Hides the content of the test suite file
Hide output block around Pytest/JUnit5Hides the output message, and individual test cases are no longer shown
Hide config block around MCQs/Select AllHides the correct answer in AutoTest
Hide output block around MCQs/Select AllHides whether the selected option is correct or not
Hide config block around Script/Custom TestHides the bash commands in the block
Hide output block around Script/Custom TestHides the output from running the blocks
{% hint style="info" %} The *Hide Result* block prevents students from seeing the outcome of tests, including whether they passed or failed and the total points achieved. When this block is enabled, students will not see any indication (such as a green tick, red cross, or score) regardless of the test results. {% endhint %} This functionality ensures that sensitive or unnecessary information is hidden from students, allowing for a more focused and secure testing environment.
*** ## 4. **Quiz and Interactive Blocks** Blocks for creating interactive quizzes and prompts for the student to engage with in the CodeGrade editor.
Quiz The *Quiz* block transforms your CodeGrade assignment into an interactive quiz, allowing students to answer questions directly in the Code Editor. This block supports various question types, making it ideal for testing both theoretical knowledge and practical coding skills.

Quiz block with different question types

**Question Types in the Quiz Block:** * Multiple Choice Question * Select All Question * Coding Question * Prompt Engineering Question (only available in our opt-in AI Beta) {% hint style="info" %} Each question type can be configured to run individually, so students can check their answers without waiting for previous questions to be graded. {% endhint %} {% hint style="success" %} Students can fill in Quizzes only by using CodeGrade's Online Editor. For an assignment that includes quizzes, it is therefore necessary to have the Online Editor as the only allowed way of creating submissions. In this case, it is also convenient to enable the [Simple Submission Mode](https://help.codegrade.com/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings), so that students are navigated to the Editor automatically when launching an assignment. {% endhint %}
Multiple Choice Question Create a *Multiple Choice Question* where students must select one correct answer from a set of options. You can easily add or remove answers using the buttons on the right, and mark the correct answer using the checkboxes on the left of each option.

Configuring a Multiple Choice Question in a Quiz

For each answer, you can provide a hint by clicking the 💡 icon. This hint will appear when the student selects that option. You can offer simple feedback such as `"Correct, well done!"` or provide more detailed explanations to help the student understand why their answer is correct or incorrect. You can configure the order in which the answers will be displayed can either be random (different for each student) or just the same as you add them. {% hint style="info" %} For more information on this block, visit [Multiple Choice Question](/automatic-grading-guides/quizzes/multiple-choice-question). {% endhint %}
Select All Question Create a *Select All Question*, where students must select all correct answers from a list of options. You can add or remove answer choices, and mark the correct ones with checkboxes.

Select All Question in a Quiz

{% hint style="info" %} A `None of the above` option is automatically added as the last choice. Selecting this option will deselect any other selected answers. {% endhint %} For each answer, you can provide two types of feedback by clicking the 💡 icon: * **Hint**: Displays when the student selects an incorrect answer. * **Feedback when Correct**: Displays when the student selects the correct answer {% hint style="info" %} For more information on this block, visit [Select All Question](/automatic-grading-guides/quizzes/select-all-question). {% endhint %}
Coding Question The *Coding Question* block allows you to assess students' programming skills by asking them to write code in response to a specific problem. This block supports a wide range of test scenarios, from simple code snippets to complex assignments requiring multiple steps. In this block, students can write their answers directly in the Online Editor, and their code can be automatically tested.

The configuration of a Coding Question

{% hint style="danger" %} When **Run Single Questions** is enabled and the student presses the `"Check Your Answer"` button for a *Coding Question*, only the current question and its associated tests will be executed. Tests that rely on setup outside the *Coding Question* block may not function as expected. {% endhint %} **Key Features:** * **Customizable Question Setup**: Define the question's name, student file name to create, description, and optionally provide a partially written code template for students to complete. You can also use markdown to format the question text and include external links to resources (e.g., Wikipedia). * **Automated Testing**: Once the student submits their code, use various test execution blocks to automatically evaluate the correctness and functionality of their solution. {% hint style="info" %} For more information on this block, visit [Coding Question](/automatic-grading-guides/quizzes/coding-question). {% endhint %}
Prompt Engineering Question The *Prompt Engineering Question* block asks students to solve problems by creating effective prompts for AI models or systems. In this question, students will write a prompt, and the system will execute it to generate results. These results will be evaluated based on their accuracy and alignment with the expected outcome.

A Prompt Engineering Question is tested by running Pytest on the generated output

**Key Features:** * **Student Input**: Students craft prompts and submit them. The output generated by the AI is saved in a separate file as specified in the block’s configuration. * **Grading Based on Output**: The prompt can assessed based on the AI's response, ensuring that the student’s prompt leads to the expected results. * **Additional System Prompts**: You can provide an additional system prompt to guide the AI’s response, enhancing the control over the output generated. This question type is useful for assignments that involve AI interactions, such as language models or code generation tools, allowing students to demonstrate their understanding of prompt crafting for AI systems. {% hint style="info" %} The AI tools are currently in Beta version. Please email for more information on joining the Beta version and getting access. {% endhint %}
*** ## 5. **Environment Management Blocks** Blocks that control the environment and file isolation during setup and test execution.
Programming Language Setup Blocks In the **AutoTest Setup** phase, CodeGrade offers predefined blocks to install and configure the required programming languages and compilers for your tests. These blocks ensure that the correct environments are set up before running any tests on each student’s submission.

Use language setup blocks in the AutoTest Setup phase

* **Java**: Installs and configures a Java environment for Java-based assignments. Multiple versions available (e.g., Java 8, Java 11, Java 17, Java 21, Java 25). * **GCC (GNU Compiler Collection)**: Installs the GCC compiler for compiling C and C++ programs. Multiple versions available (e.g., GCC 9, GCC 10, GCC 13). * **Python**: Installs the required version of Python for Python-based assignments. Multiple versions available (e.g., Python 3.10, Python 3.11, Python 3.12, Python 3.13). * **Clang**: Installs the Clang compiler for compiling C and C++ programs, often used as an alternative to GCC. Multiple versions available (e.g., Clang 12, Clang 16). * **Node.js**: Installs a specified version of Node.js for JavaScript-based assignments. Multiple versions available (e.g., Node 14, Node 16, Node 18, Node 24). * **.NET**: Installs the required version of the .NET framework for running C# or other .NET-based assignments. Multiple versions available (e.g., .NET 8.0, .NET 9.0). * **R:** Installs the required version of **R** for running R-based assignments. Initial support available (R 4.5). {% hint style="info" %} If the available language setup blocks do not meet your needs, the [*Script*](#script) block offers the flexibility to run custom bash commands to install and configure any environment you require for your tests. {% endhint %}
Isolate The *Isolate* block ensures that the blocks nested within it run in a controlled, isolated environment. Once the block completes, the original state of the file system and environment is restored, preventing any changes from affecting subsequent tests.

Compiled Java program executed in isolation from other tests

**Example Use Cases:** * **Temporary Logs or Configuration Files**:\ If a student's program generates temporary logs or modifies configuration files, wrapping these operations in an *Isolate* block ensures that any changes to the file system are contained. After the block finishes, the environment is restored, eliminating any unintended side effects on later tests. * **Modifying Data Files**:\ When a student's code modifies a data file (e.g., CSV or JSON), the *Isolate* block ensures that any changes are rolled back after the test. This guarantees that the file remains unchanged for other tests, preserving the integrity of the data.
*** ## 6. **Resource and File Management Blocks** Blocks related to uploading, managing, and handling files during tests.
Upload Files The *Upload Files* block allows you to upload files to be used within your test blocks. Files are placed in the `$UPLOADED_FILES` directory.

Upload a Java program using Upload Files block

You can use this block to upload scripts, data files, or test fixtures needed for your automated tests. It also allows you to create, edit, rename, or delete files directly within the block. To download a file, simply click its name and use the download button in the top-right corner of the file editor.

View, edit, or download the uploaded files

{% hint style="info" %} The `$UPLOADED_FILES` directory is separate from the working directory in the Test phase of AutoTest. Ensure that you move or copy uploaded files to the current working directory, or reference them using their full path to avoid issues during testing. {% endhint %}
Output The *Output* block enables you to upload files or directories to the student's AutoTest results.

Output the specified generated file to the Submission Overview

**Use Case:** When student submissions generate output files or directories (e.g., logs, reports), you can capture and display these files alongside the student's code in the submission overview. **How It Works:** 1. **Capture Output**: Use a *Script* block to run a bash command that runs the student's program and generate output files. 2. **Upload Files**: The *Output* block uploads the captured files to Submission Overview. Ensure the filenames match the ones referenced in the code, or use a blob pattern to upload specific types of output files. This block helps present relevant output files, such as logs or data files, in the submission review, making it easier to evaluate the student’s work.
*** By understanding and utilizing the diverse blocks in AutoTest V2, you can automate and customize the grading process for a wide variety of assignments. This allows for greater efficiency, flexibility, and accuracy in evaluating student submissions. From setting up programming environments to managing test execution and feedback, these blocks give you complete control over how assignments are graded. With the insights provided in this guide, you'll be well-equipped to configure, optimize, and scale your automatic grading systems to meet your specific needs. # Quizzes You can transform your CodeGrade assignment into an Interactive Quiz with the Quiz block in AutoTest. This allows you to create several different types of questions that students can fill in in the online editor. ## Multiple Choice Question {% content-ref url="/pages/l4jPG0xfcyFFDgKT7Day" %} [Multiple Choice Question](/automatic-grading-guides/quizzes/multiple-choice-question) {% endcontent-ref %} ## Select All Question {% content-ref url="/pages/sv7k0F7y9MMSDzKHzYoC" %} [Select All Question](/automatic-grading-guides/quizzes/select-all-question) {% endcontent-ref %} ## Coding Question {% content-ref url="/pages/bSq8WEMzVIlBTj4HG4wg" %} [Coding Question](/automatic-grading-guides/quizzes/coding-question) {% endcontent-ref %} # Multiple Choice Question Sometimes, programming assignments aren't the most efficient way of testing students' knowledge of programming theory. In those situations, Multiple Choice Questions (MCQs) may be more practical. The Multiple Choice Question block allows you to set up a fully automatically graded MCQ in your assignments. {% hint style="success" %} Students can fill in Quizzes only by using CodeGrade's Online Editor. For an assignment that includes quizzes, it is therefore necessary to have the Online Editor as the only allowed way of creating submissions. In this case, it is also convenient to enable the [Simple Submission Mode](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings), so that students are navigated to the Editor automatically when launching an assignment. {% endhint %} ### Create a Multiple Choice Question Test To create a MCQ, first add a Quiz Block, and then include a Multiple Choice Question Block within it. In a Multiple Choice Question Block, you can set: * The name of the Block so that the student knows what the question is about; * A description of the question, by filling in the **Question** field. This field supports markdown so that you can format the text or include links to external resources, such as in the following example: ```markdown Which of the following functions prints the first `n` [perfect squares](https://en.wikipedia.org/wiki/Perfect_square) ``` Let's see more in detail how to set answers: * New answers can be added by clicking the :heavy\_plus\_sign: icon on the right of an answer block; * The order in which the answers will be displayed can either be random (different for each student) or just the same as you add them; * You will have to select which answer is the correct one by clicking the circle on the left of the corresponding block; * Each answer has its own markdown description field. Here, you can add formatted text or code snippets, as in the following example: ````markdown ```python def print_first_n_squares(n): for i in range(1, n + 1): print(i * i) ``` ```` * for each answer, you can provide a hint, again in Markdown, by clicking the :bulb: icon. This will be displayed if the student chooses that answer. This could be a simple confirmation message like `Correct, well done!` , or you could provide feedback to help the student understand why the selected answer is wrong.

The configuration of a Multiple Choice Question in AutoTest.

### Workflow suggestions: #### Hiding information from the student It is generally good practice to use **Hide Blocks** with Multiple Choice Questions, as students, by default, can see: * Whether their answer is right or wrong. You can use a **Hide Result Block** to prevent this (hints will be masked too); * The correct answer, by inspecting their own submission after handing in. You can use a **Hide Configuration Block** to prevent this. Remember that any hidden information can be released to the students after the deadline or lock date! #### Limited amount of submissions Also, with Quizzes, you can [limit the number of submissions](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings) that the student can make. This can be especially useful for MCQs when nesting them within a **Run Only In Block.** The **Run Only In Block,** when set to **Submission**, will force the student to spend a submission attempt in order to receive feedback about their MCQs. #### Example To prevent the students from answering a MCQ with a brute force approach, we may then, for example: * Wrap a **Run Only in Submission Block** around our Quiz Block; * Wrap a **Hide Configuration Block** around our MCQ Block; * Set the number of submission attempts to a small number like 2. In this way, the student will be informed whether their answer is correct only after submitting, and, if needed, will have just one more attempt to guess the correct answer.

The Select All Question configuration discussed in the example.

{% hint style="success" %} To make Quizzes graded, remember to wrap a Connect Rubric Block around them. {% endhint %} {% hint style="success" %} Quizzes can be used in combination with Run-If blocks. You may, for example, run other tests conditionally on the result of a Quiz. {% endhint %} ### AutoTest Snapshot and Test Student Submission With your Multiple Choice Question Block now set up, click the bottom right **Build Snapshot** button to build your AutoTest Snapshot. The first time you build a snapshot, you are required to upload a Test Submission; in this case it will be built for you from the solutions filled in the AutoTest setup. Simply click to generate a Test Submission:

Automatic Test Submission generation.

After publishing your snapshot, the student can fill in the Multiple Choice Question and receive feedback in the Online Editor as shown below:

An example of the feedback the student receives when filling in a Multiple Choice Question in the Editor.

# Select All Question Sometimes programming assignments aren't the most efficient way of testing students' knowledge of programming theory. In those situations, a Select All Question may be more practical. In a Select All Question, a student is presented with multiple possible answers. Unlike in a [Multiple Choice Question](/automatic-grading-guides/quizzes/multiple-choice-question), there may be a single correct answer, multiple correct answers, or none of the answers may be correct. {% hint style="success" %} Students can fill in Quizzes only by using CodeGrade's Online Editor. For an assignment that includes quizzes, it is therefore necessary to have the Online Editor as the only allowed way of creating submissions. In this case, it is also convenient to enable the [Simple Submission Mode](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings), so that students are navigated to the Editor automatically when launching an assignment. {% endhint %} ### Create a Select All Question To create a Select All Question, first add a Quiz Block, and then include a Select All Question Block within it. Configuring a Select All Question is very similar to configuring a [Multiple Choice Question](/automatic-grading-guides/quizzes/multiple-choice-question), with some important differences: * You are not required to select at least one correct answer; * You can select as many correct answers as you want; * An extra **None of the above** option is automatically added as the last option, which, when selected, will deselect all other options selected by the student; * For each answer, by clicking the :bulb: icon, you can provide two kinds of feedback: * a `Hint`, that will be displayed only when the student uncorrectly selects the answer. This happens when the current question is not correct, and the student selects it, or vice versa; * a `Feedback when correct` message, which will be displayed only when the student correctly selects the answer.

For a Select All Question, you can set multiple or zero correct answers and provide feedback depending on the choice of the student.

### Workflow suggestions: #### Hiding information from the student It is generally good practice to use **Hide Blocks** with Select All Questions, as students, by default, can see: * Whether their answer is right or wrong. You can use a **Hide Result Block** to prevent this (hints will be masked too); * The correct answer(s), by inspecting their own submission after handing in. You can use a **Hide Configuration Block** to prevent this. Remember that any hidden information can be released to the students after the deadline or lock date! #### Limited amount of submissions Also, with Quizzes, you can [limit the number of submissions](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings) that the student can make. This can be especially useful for Select All Questions when nesting them within a **Run Only In Block.** The **Run Only In Block,** if set to **Submission**, will force the student to spend a submission attempt in order to receive feedback about their Select All Questions. #### Example To prevent the students from answering with a brute force approach, we may then, for example: * Wrap a **Run Only in Submission Block** around our Quiz Block; * Wrap a **Hide Configuration Block** around our Select All Question Block; * Set the number of submission attempts to a small number like 2. In this way, the student will be informed whether their answer is correct only after submitting, and, if needed, will have just one more attempt to guess the correct answer.

The Select All Question configuration discussed in the example.

{% hint style="success" %} To make Quizzes graded, remember to wrap a Connect Rubric Block around them. {% endhint %} {% hint style="success" %} Quizzes can be used in combination with Run-If blocks. You may, for example, run other tests conditionally on the result of a Quiz. {% endhint %} ### AutoTest Snapshot and Test Student Submission With your Select All Questions Block now set up, click the bottom right **Build Snapshot** button to build your AutoTest Snapshot. The first time you build a snapshot, you are required to upload a Test Submission; in this case it will be built for you from the solutions filled in the AutoTest setup. Simply click to generate a Test Submission:

Automatic Test Submission generation.

After publishing your snapshot, the student can fill in the Select All Question and receive feedback in the Online Editor as shown below:

An example of the feedback the student receives when filling in a Select All Question in the Online Editor.

# Coding Question To assess students' understanding of programming concepts effectively, creating specific questions that require them to produce code snippets can be very beneficial. A **Coding Question** follows this precise workflow: students read the question and then write a code snippet as their answer in the Online Editor. As a teacher, you can subsequently run automatic tests on the code produced by the students to evaluate their solutions. {% hint style="success" %} Students can fill in Quizzes only by using CodeGrade's Online Editor. For an assignment that includes quizzes, it is therefore necessary to have the Online Editor as the only allowed way of creating submissions. In this case, it is also convenient to enable the [Simple Submission Mode](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings), so that students are navigated to the Editor automatically when launching an assignment. {% endhint %} ### Create a Coding Question Test To create a Coding Question, first add a Quiz Block and then include a Coding Question Block within it. In a Coding Question Block, you can set: * The name of the Block so that the student knows what the question is about; * A description of the question, by filling in the **Question** field. This field supports markdown so that you can format the text or include links to external resources, such as in the following example: ```markdown Complete the following code snippet so that it prints the first 20 [prime numbers](https://en.wikipedia.org/wiki/Prime_number). ``` * The name of the file that the student will produce; * A **Starter Code Template** that the student will start working with. For instance, you may provide a partially filled code snippet, as in the example below: ```python def is_prime(num): if num < 2: return ___ for i in range(2, ___): if num % i == ___: return False return ___ prime_numbers = [] current_number = ___ while len(prime_numbers) < ___: if is_prime(___): prime_numbers.append(current_number) current_number += 1 print(prime_numbers) ``` * a **Solution,** which can be used to create a Test Submission, and can be revealed to students.

The configuration of a Coding Question.

After the student submits their answer, you can evaluate their code using the full capabilities of AutoTest. Within the Coding Question framework, you have access to all the available Test Blocks that facilitate grading workflows. These are detailed in our guides for different programming languages, like our [Python](/automatic-grading-guides/python) or [Java ](/automatic-grading-guides/java)guide for example. The image below shows an example where, for a Python assignment, an IO Test Block is used to check that the student's code snippet produces the expected output:

An Input Output test is used to test the student's code snipper for a Coding Question.

### Hiding information from the student As explained in the guides for the different programming languages, teachers can select which information about a Test Block should be revealed or hidden from the student. The same holds for Test Blocks that are part of a Coding Question: it will be sufficient to use Hide Blocks just as with regular Test Blocks! This is especially important to hide the solution. {% hint style="info" %} To make Quizzes graded, remember to wrap a Connect Rubric Block around them. {% endhint %} {% hint style="info" %} Also, with Quizzes, you can [limit the number of submissions](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings) that the student can make. {% endhint %} {% hint style="info" %} Quizzes can be used in combination with Run-If blocks. You may, for example, run other tests conditionally on the results of a Quiz. {% endhint %} ### AutoTest Snapshot and Test Student Submission With your Coding Question Block now set up, click the bottom right **Build Snapshot** button to build your AutoTest Snapshot. The first time you build a snapshot, you are required to upload a Test Submission; in this case it will be built for you from the solutions filled in the AutoTest setup. Simply click to generate a Test Submission:

Automatic Test Submission generation.

After building and publishing your snapshot, the student can fill in the Coding Question and receive feedback in the Online Editor as shown below: # Python ### Create your first autograded Python assignment {% content-ref url="/pages/jj72ts0v49Yks7MZ2EGa" %} [Create your first Python assignment](/automatic-grading-guides/python/create-your-first-python-assignment) {% endcontent-ref %} ### Advanced grading options for Python assignments {% content-ref url="/pages/TG4hYAqT08ERPDOo4QUy" %} [Advanced Python autograding](/automatic-grading-guides/python/advanced-python-autograding) {% endcontent-ref %} ### # Create your first Python assignment Create your first autograded Python assignment In this tutorial we will guide you through all the steps to create your very first Python assignment in CodeGrade. For this tutorial we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/vzM9cB9cRQDVgH7BNlJU" %} **A quick note about this example**\ In this assignment, students have been asked to create a Python program that prints the first 'n' numbers of the Fibonacci sequence, where 'n' is determined by user input. ## Step 1: Create an assignment First, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.
3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Fibonacci.zip folder and upload Fibonacci/template/fibonacci.py as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [submission settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric categories: * **Discrete category** - Specify points in discrete steps (eg. 0, 5, or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create New Rubric**. 3. Create 2 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
IO testsCheck that your program produces the expected output given a specific input.0 - 75
Code structure testsCheck that your program uses the required code structures.0 - 25
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allows them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial, we will be setting up two types of automatic tests: IO tests and Semgrep code structure tests. ### Step 4.1: Setup

An Install Python block in the Setup of AutoTest v2

Before actually creating the tests, we need to make sure we have the correct version of Python installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add an **Install Python** block to your configuration. 3. Select Python version 3.11 to install. ### Step 4.2: Input/Output tests
Input/Output tests (or "IO tests" for short) are a great way of checking a program's functionality by providing various input cases that should result in different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Navigate to the **Tests** tab of the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration. Select the "IO tests" rubric category from the drop-down menu. 3. Add an **IO test** block to your AutoTest configuration and nest it inside the Connect Rubric block. 4. In the editor provided use the command `python fibonacci.py` to run the program. 5. Add five **Substring match** blocks to your AutoTest configuration and nest them inside of the IO test block. Make sure each match block is case insensitive and ignores whitespace. 6. Copy and paste the info from the table below into the **Input** and **Expected output** fields in their respective match block: | Input | Expected output | | ----- | ---------------------------- | | 1 | 1 | | 3 |

1
1
2

| | 5 |

1
1
2
3
5

| | 0 | invalid | | -1 | invalid | {% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to both you and your students what is actually being checked (e.g. "Does your program produce the correct output? - Input: 5"). {% endhint %} ### Step 4.3: Code structure tests

A graded Code structure test in AutoTest v2

Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax to reach their solution. AutoTest v2 offers the **Code structure test** block and its corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example, we want to check that the students have used a while loop and an if-elif-else statement in their program. Here are the Semgrep rules we will be using: {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: | while $COND: ... message: Semgrep found a match languages: [python] severity: WARNING ``` {% endcode %} {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: | if $COND1: ... elif $COND2: ... else: ... message: Semgrep found a match languages: [python] severity: WARNING ``` {% endcode %} 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "fibonacci.py". 4. Add two **Positive match** blocks to your test configuration and nest them within the Code structure test block 5. Copy and paste the semgrep rules in the code blocks above into the provided editor in each Positive match block. Make sure to delete the template provided. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the `Fibonacci/Solution/fibonacci.py` file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit" Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish snapshot** button. Once it is published, AutoTest V2 is available for students through the editor and on the submission page. ## Conclusion You have just built your first automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](/faq) or reach out to our support team at . # Advanced Python autograding Discover the advanced autograding options available for Python assignments In this guide, we explore the advanced grading options available for Python assignments. For more information about setting up a Python assignment from scratch, see: {% content-ref url="/pages/jj72ts0v49Yks7MZ2EGa" %} [Create your first Python assignment](/automatic-grading-guides/python/create-your-first-python-assignment) {% endcontent-ref %} ## Simple Python Test
The simple Python test block is a great tool for building assignments geared towards beginner programmers. It allows you to insert your students' code into a Python script, enabling more grading and feedback options than just IO testing. ### Example 1 One use of the Simple Python Test block is to ask students to write basic Python code without writing a complete program. Consider the example submission below: {% code title="reversed\_sentence.py" %} ```python ''' Assume that a string variable called "sentence" has already been declared for you. Write a statement using string slicing that reverses the string and saves it to a variable called "reversed_sentence". ''' reversed_sentence = sentence[::-1] ``` {% endcode %} As you can see, this program would not be able to run on its own since `sentence` is not defined. Using the Simple Python Test, we can define the variable and insert the student's code to make a complete program. The student's code is inserted in the Simple Python Test block using the comment `# CG_INSERT filename`. ```python sentence = "This is another sentence" answer = sentence[::-1] # CG_INSERT reversed_sentence.py print("your answer was: ", reversed_sentence) print("correct answer is: ", answer) if reversed_sentence == answer: "Your answer was correct!" exit(0) else: "Your answer was incorrect!" exit(1) ``` ### Example 2 Another use case for the Simple Python Test block is to provide better feedback to students using the `cg_feedback_helpers` Python package. Consider the example submission below: {% code title="bubble\_sort.py" %} ```python ''' Assume you have been provided with a list of integers called `numbers`. Sort the list from smallest to largest using the bubble sort method. ''' for i in range(length - 1): for j in range((length - 1) - i): if numbers[j] > numbers[j + 1]: numbers[j], numbers[j + 1] = numbers[j + 1], numbers[j] ``` {% endcode %} Using the `cg_feedback_helpers` module with the Simple Python Test block we can run our students' code in the context of a full program and use assertions to provide better feedback. The `cg_feedback_helpers` package can be installed in the AutoTest setup using the command `python -m pip install cg_feedback_helpers`. ```python from cg_feedback_helpers import asserter, NO_FEEDBACK numbers = [34, 12, 99, 54, 28] sorted_numbers = [12, 28, 34, 54, 99] # CG_INSERT bubble_sort.py print("your answer was: ", numbers) print("correct answer is: ", sorted_numbers) # Assert the answer is correct asserter.equals( numbers, sorted_numbers, positive_feedback = "Your answer was correct!", negative_feedback = "Your answer was incorrect!" ) # Display the feedback asserter.emit_success(feedback=NO_FEEDBACK) ```
## Pytest
[Pytest](https://docs.pytest.org/) is the industry standard unit testing framework for Python. The Pytest block is a dedicated test block for running Pytest unit tests on your students' submissions. This block is great for grading assignments in which students create their own functions. Consider the example submission below: 
'''
    Write a function for the arithmetic operations add, subtract, multiply, and divide.
    Each function should have two arguments and should return the correct answer.
    The divide function should raise the ValueError exception if an attempt is made to divide by 0.
'''

def add(x, y):
    ans = x + y
    return ans


def subtract(x, y):
    ans = x + y
    return ans


def multiply(x, y):
    ans = x * y
    return ans

def divide(x, y):
    if y == 0:
        raise ValueError("Can not divide by zero!")
    ans = x / y
    return ans
Using the Pytest block, we can easily create multiple test cases to assess the functionality of the `square()` function. Additionally, we can use the `cg_pytest_reporter` PyPi module to enhance the feedback from our test cases. The `@name` and `@description` decorators allow us to make the test cases clearer and more readable for students. The `@weight` decorator allows us to adjust the grading weight of separate test cases. ```python import pytest from calculator import add, subtract, multiply, divide from cg_pytest_reporter import name, description, weight @name("Test add()") @description("Check that add() produces the correct result") def test_add(): assert add(2,3) == 5 assert add(1,-1) == 0 assert add(-1,-1) == -2 @name("Test subtract()") @description("Check that subtract() produces the correct result") def test_subtract(): assert subtract(5, 2) == 3 assert subtract(1, -1) == 2 assert subtract(-1, -1) == 0 @name("Test multiply()") @description("Check that multiply() produces the correct result") def test_multiply(): assert multiply(3, 2) == 6 assert multiply(1, -1) == -1 assert multiply(-1, -1) == 1 @name("Test divide()") @description("Check that divide() produces the correct result or raises an error") @weight(2) def test_divide(): assert divide(10, 2) == 5 assert divide(1, -1) ==-1 assert divide(-1, -1) == 1 assert divide(5, 2) == 2.5 with pytest.raises(ValueError): divide(10, 0) ``` The feedback from each test case is also beautifully rendered, making it easy for students to read.
## Flake8
[Flake8](https://pypi.org/project/flake8/) is an industry-standard linter for Python that enforces the [PEP8 style guide](https://peps.python.org/pep-0008/). It is a useful tool for enforcing code styling best practices for beginner programmers. The Flake8 block is an off-the-shelf testing tool that doesn't require any additional setup or configuration other than providing the name of the file to be tested. The Flake8 block generates comments directly on students' code. Each target line is highlighted according to the severity of the comment and the comment can be read by hovering over the line number with the mouse cursor. The Flake8 block also works in the built-in editor making it a powerful combination for beginners.
You can also update the configuration of Flake8 to ignore or select certain code style rules. This can be done by editing the **config** field. For example, this configuration would ignore the E203 error (Whitespace before ','.) as defined by the `pycodestyle` package: ``` [flake8] extend-ignore = E203 ``` ## Conclusion These advanced Python testing options open the door to more rigorous testing methods and better student feedback. However, these are just the readily available code blocks. There are many more testing tools and packages available that you can use with some simple setup. For more information, contact us at . # Java ### Create your first autograded Java assignment {% content-ref url="/pages/3SWN5XkkYLx5GarvlxWh" %} [Create your first Java assignment](/automatic-grading-guides/java/create-your-first-java-assignment) {% endcontent-ref %} ### Advanced grading options for Java assignments {% content-ref url="/pages/ShkUZHTEvqf8R1vCFGBI" %} [Grading with JUnit5 and Checkstyle](/automatic-grading-guides/java/grading-with-junit5-and-checkstyle) {% endcontent-ref %} {% content-ref url="/pages/U9C3dpTPVQzNitCbtHAN" %} [Grading with Maven](/automatic-grading-guides/java/grading-with-maven) {% endcontent-ref %} {% content-ref url="/pages/2mNiVzRQHRZgP1XZ5QNp" %} [Grading Java package projects](/automatic-grading-guides/java/grading-java-package-projects) {% endcontent-ref %} {% content-ref url="/pages/OAAFhJE0fI7c7Xou7ukP" %} [Grading test coverage with JaCoCo](/automatic-grading-guides/java/grading-test-coverage-with-jacoco) {% endcontent-ref %} # Create your first Java assignment Create your first automatically graded Java assignment using CodeGrade's AutoTest v2 In this tutorial we will guide you through all the steps to create your very first Java assignment in CodeGrade. For this tutorial we will be considering the example below. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/M2RvIiRJvE5YOVXN8vEL" %} #### A quick note about this example In this assignment, students have been asked to create a Java program that prints the first 'n' numbers of the Fibonacci sequence, where 'n' is determined by user input. ## Step 1: Create an assignment First, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.
3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Fibonacci.zip folder and upload Fibonacci/Template/Fibonacci.java as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [Submission Settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric
While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric category: * **Discrete category** - Specify points in discrete steps (eg. 0, 5 or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 3 new **Continuous Categories**. Set the parameters as follows:
NameDescriptionPoints
Compile testCheck that your code compiles without errors.0 - 20
Input/Output testsCheck that your program produces the correct output depending on the provided input.0 - 40
Code structure testsCheck that you have used the expected code structures0 - 40
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial, we will be setting up several automatic tests: Input/Output tests and Semgrep code structure tests. ### Setup 4.1: Setup

An Install Java block in the Setup of AutoTest v2

Before actually creating the tests, we need to make sure that the correct version of Java is installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add an **Install Java** block to your configuration and select the latest version of Java from the drop-down menu. ### Step 4.2: Compile test
As with any compiled language like Java, each student's program must be compiled before we can run any other tests on it. We will also grade this test using the "Compile test" rubric category. In this example we will go one step further than just showing the compiler errors. Instead we will parse the compiler output and if errors occur, we will place comments on the students code at the line that the error occurred on. This is done using the **Java Compile** block. 1. Navigate to the **Tests** tab under the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration and select the "Compile test" rubric category from the drop-down menu. 3. Add a **Java Compile** block to your AutoTest configuration and nest it within the **Connect rubric** block. 4. In the editor provided, add the file or glob pattern to compile: ```sh $STUDENT/Fibonacci.java ``` ### Step 4.3: Input/Output tests Input/Output tests (or IO tests for short) are a great way of checking a program's functionality by providing various input cases that should result in different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks.
1. Add a **Connect Rubric** block to your AutoTest configuration and select the "IO tests" rubric. 2. Add an **IO test** block to your AutoTest configuration and nest it in the Connect Rubric block. Run the compiled java file by writing the command `java Fibonacci` in the provided editor. 3. Add five **Substring match** blocks to your AutoTest configuration and nest them inside of the IO test block. 4. Copy and paste the info from the table below into the **Input** and **Expected output** fields in their respective match block: | Input | Expected output | | ----- | ---------------------------- | | 1 | 1 | | 3 |

1
1
2

| | 5 |

1
1
2
3
5

| | 0 | invalid | | -1 | invalid | ### Step 4.4: Code structure tests
Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax for reaching their solution. AutoTest v2 offers the **Code structure test** block and it's corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example we want to check that the students have used a for loop as part of their solution. Here is the Semgrep rule we will be using to check for this loop: {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: | for (...) { ... } message: Semgrep found a match languages: [java] severity: WARNING ``` {% endcode %} 1. Add a **Connect rubric** block to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code structure test** block to your test configuration and nest it within the **Connect rubric** block. 3. In the **Student file** input field, write "Fibonacci.java". 4. Add a **Positive match** block to your test configuration and nest it within the Code structure test block 5. Copy and paste the semgrep rule in the code block above into the provided editor. Make sure to delete/replace the template provided in the editor. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the Fibonacci/Solution/Fibonacci.java file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit". Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish to students** button. ## Conclusion You have just built your first automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](/faq) or reach out to our support team at . # Grading with JUnit5 and Checkstyle Discover the advanced autograding options available for Java assignments In this guide, we explore the advanced grading options available for Java assignments. For more information about setting up a Java assignment from scratch, see {% content-ref url="/pages/3SWN5XkkYLx5GarvlxWh" %} [Create your first Java assignment](/automatic-grading-guides/java/create-your-first-java-assignment) {% endcontent-ref %} ## JUnit5 Junit5 is an industry-standard unit testing framework for Java. It is particularly useful for grading assignments that require students to write functions and classes. Using Junit5 unit tests offers several advantages over conventional IO tests including the ability to use assertions, parametrize test cases, and provide better feedback for students. Consider the example submission below: {% code title="Calculator.java" %} ```java import java.util.Scanner; public class Calculator { public static void main(String[] args) { Scanner sc = new Scanner(System.in); System.out.print("Select operator: "); String operator = sc.nextLine(); System.out.print("Enter two numbers: "); int num1 = sc.nextInt(); int num2 = sc.nextInt(); if (operator.equals("add")) { add(num1, num2); } else if (operator.equals("subtract")) { subtract(num1, num2); } else if (operator.equals("multiply")) { multiply(num1, num2); } else if(operator.equals("divide")) { divide(num1, num2); } else { System.out.println("The method you entered doesn't exist"); } } public static float add(float numberOne, float numberTwo) { System.out.println(numberOne + numberTwo); return numberOne + numberTwo; } public static float subtract(float numberOne, float numberTwo) { System.out.println(numberOne - numberTwo); return numberOne - numberTwo; } public static float multiply(float numberOne, float numberTwo) { System.out.println(numberOne * numberTwo); return numberOne * numberTwo; } public static float divide(int numberOne, int numberTwo) { System.out.println(numberOne / numberTwo); return numberOne / numberTwo; // Should return ArithmeticException } } ``` {% endcode %} We can create robust test cases to check this submission using Junit5 assertions. Additionally, we can use decorators such as `@displayName` to make our test cases more readable and to increase the weight of individual test cases. {% code title="TestCalculator.java" %} ```java import java.util.*; import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.DisplayName; public class TestCalculator { @Test @DisplayName("Check whether the addition function returns the expected result.") public void test_add() { Assertions.assertEquals( 5, Calculator.add(2, 3), 0.2); Assertions.assertEquals( 0, Calculator.add(1, -1), 0.2); Assertions.assertEquals(-2, Calculator.add(-1, -1), 0.2); } @Test @DisplayName("Check whether the subtraction function returns the expected result") public void test_subtract() { Assertions.assertEquals( 3, Calculator.subtract(5, 2), 0.2); Assertions.assertEquals( 2, Calculator.subtract(1, -1), 0.2); Assertions.assertEquals( 0, Calculator.subtract(-1, -1), 0.2); } @Test @DisplayName("Check whether the multiply function returns the expected result") public void test_multiply() { Assertions.assertEquals( 6, Calculator.multiply(2, 3), 0.2); Assertions.assertEquals(-1, Calculator.multiply(1, -1), 0.2); Assertions.assertEquals( 1, Calculator.multiply(-1, -1), 0.2); } @Test @DisplayName("[2] Check whether the divide function returns the expected result and throws an ArithmeticException when dividing by 0 (weight 2)") public void test_divide() { Assertions.assertEquals( 5, Calculator.divide(10, 2), 0.2); Assertions.assertEquals(-1, Calculator.divide(1, -1), 0.2); Assertions.assertEquals( 1, Calculator.divide(-1, -1), 0.2); Assertions.assertThrows(ArithmeticException.class, () -> { Calculator.divide(10, 0); }); } } ``` {% endcode %} ### Junit5 Block To make running JUnit5 tests in CodeGrade simpler, we created the **Junit5** block. This can be used to install JUnit5, automatically compile and run your JUnit5 test cases, or fetch the JUnit5 Jar files to run JUnit5 with your own custom specifications. The block then renders the test results beautifully and makes it easy to read and interpret, as shown in the image below.
### Instructions
1. In the AutoTest settings, navigate to the **Tests** tab. 2. Add a **Connect Rubric** Block and a **Junit5** block to your AutoTest configuration. Nest the Junit5 block within the Connect rubric block. 3. Copy the contents of your JUnit 5 test file (e.g., `TestCalculator.java`) and paste them into the Junit5 block's editor. 4. Ensure that the Test file name matches the class name in the editor. 5. Build and publish your snapshot {% hint style="info" %} JUnit5 has the known issue that it isn't possible to specify the order in which test cases are run. As a result, the results for each test case may displayed in a seemingly random order. {% endhint %} ## Checkstyle Checkstyle is an industry-standard linter for Java that allows you to enforce rules from various style guides, including the Sun and Google style guides. It is a useful tool for enforcing code styling best practices for beginner programmers.
The **Checkstyle block** is an off-the-shelf testing tool that doesn't require any additional setup or configuration other than providing the name of the file to be tested. The Checkstyle block generates comments directly on students' code. Each target line is highlighted according to the severity of the comment and the comment can be read by hovering over the line number with the mouse cursor. The Checkstyle block also works in the built-in editor making it a powerful combination for beginners.
Although no setup is required, you can select from three readily available style guides: Checkstyle's default style guide, Sun style guide, and Google style guide. In addition, you can also update the configuration for each of these options to ignore or include any of the rules. Aside from style guides, you can also adjust the percentage of points deducted for each comment placed on a submission depending on its severity. ## Conclusion These advanced Java testing options open the door to more rigorous testing methods and better student feedback. However, these are just the most commonly used methods. There are many more testing tools and packages available that you can use with some simple setup. For more information, contact us at . # Grading with Maven ### Overview This guide shows how to set up a Maven-based Java programming assignment in CodeGrade, using instructor-written JUnit tests and AutoTest for automated grading. *** ### Configuring AutoTest Setup #### Install Java and Maven In **AutoTest > Setup**: * Add the **Install Java** block (or use the script below). * Add a **Script** block to install Java 17, Maven 3.8.4, and `junitparser`: {% code lineNumbers="true" %} ```bash sudo apt update sudo apt install openjdk-17-jre -y wget https://archive.apache.org/dist/maven/maven-3/3.8.4/binaries/apache-maven-3.8.4-bin.tar.gz -P /tmp/ sudo tar xf /tmp/apache-maven-3.8.4-bin.tar.gz -C /opt sudo ln -s /opt/apache-maven-3.8.4/ /opt/maven sudo rm /usr/bin/mvn sudo ln -s /opt/maven/bin/mvn /usr/bin/mvn pip3 install junitparser ``` {% endcode %}

AutoTest Setup for Maven assignments

*** ### Configuring AutoTest Tests #### 1. Upload and prepare instructor tests * Upload your test files using an **Upload Files** block (e.g., `CircleTest.java`). * Use a **Script** block to move them: ```bash mkdir -p src/test/java/com/example/shapes cp $UPLOADED_FILES/*.java src/test/java/com/example/shapes ``` {% hint style="warning" %} Block student-submitted test files in **General Settings > Hand-in Requirements** (e.g., deny `*Test.java`). {% endhint %}

Upload and prepare tests

#### 2. Prepare build and classpath Add a **Script** block to compile and collect dependencies: ```bash #!/usr/bin/env bash set -euo pipefail mvn -q -DskipTests=true clean package mvn -q dependency:copy-dependencies -DoutputDirectory=target/dependency -DincludeScope=test DEP_JARS="$(ls target/dependency/*.jar 2>/dev/null | tr '\n' ':' || true)" SELF_JAR="$(ls target/*.jar 2>/dev/null | tr '\n' ':' || true)" JAR_PATHS="${DEP_JARS}${SELF_JAR}target/classes:target/test-classes" CG_CP="$(cg junit5 get-classpath || true)" echo "export CLASSPATH=\"${JAR_PATHS}${CG_CP:+:${CG_CP}}:.\"" >> ~/.cg_bash_env echo 'export MAIN_CLASS="com.example.Main"' >> ~/.cg_bash_env ``` {% hint style="info" %} This ensures all compiled classes and dependencies are available to the test phase. {% endhint %} #### 3. Running Maven tests Use **Connect Rubric** + **Custom Test** blocks for each test class. Each test block can link to a separate rubric category for grading. ```bash mvn -q -Dtest=CircleTest test || true cg junitxml target/surefire-reports/TEST-CircleTest.xml ``` {% hint style="info" %} The project is compiled during the setup phase. Maven automatically skips recompilation when there are no changes. {% endhint %}

Running Maven tests

*** ### Troubleshooting Maven Autograding in CodeGrade This section lists common issues you may encounter when setting up Maven-based Java assignments in CodeGrade, along with quick solutions.
Maven or Java not found **Symptom:** AutoTest logs show `mvn: command not found` or `java: command not found` **Fix:** * Ensure Java and Maven are installed in the Setup phase. * Double-check the setup script includes the correct symbolic links to Maven: ```bash sudo ln -s /opt/maven/bin/mvn /usr/bin/mvn ```
Tests not detected **Symptom:** Maven runs without executing any tests. **Fixes:** * Ensure test class names match `*Test.java` or `*Tests.java`. * Confirm test files are copied into `src/test/java/...` * If tests use packages, ensure folders match package structure. * Use the `-Dtest=ClassName` flag to target specific test classes in test blocks.
Test failures without output **Symptom:** AutoTest runs but doesn’t show detailed feedback. **Fix:** * Verify `cg junitxml target/surefire-reports/*.xml` is included after `mvn test`. * Ensure the Surefire plugin outputs XML (default behavior). * Avoid cleaning (`mvn clean`) between test blocks unless needed.
Classpath errors or missing dependencies **Symptom:** `ClassNotFoundException` or build errors for imported classes. **Fixes:** * Run `mvn dependency:copy-dependencies -DincludeScope=test` in Setup. * Confirm your `pom.xml` includes required dependencies. * Use the environment setup script to populate `CLASSPATH`.
Timeouts or hanging tests **Symptom:** Tests hang or AutoTest fails due to long execution. **Fixes:** * Use JUnit timeouts (e.g., `@Test(timeout = 2000)` or `assertTimeout()` for JUnit 5). * Review student code for infinite loops or blocking I/O. * Set step-level timeouts in AutoTest test block settings.
#### Still Need Help? For assistance with custom cases, reach out to and share a relevant snapshot or example submission. # Grading Java package projects More advanced Java assignments often ask students to organise their code into packages. When a student's code lives under a package such as `myapp`, any instructor tests you want to run against it must sit inside the same package folder so that `javac` and JUnit can resolve the imports correctly. This guide walks through the pattern: upload the instructor's JUnit 5 test file, compile the student's code, move the test into the student's package folder, and run it. ### Step 1: Install JUnit 5 Before setting up your tests, you need to install JUnit5 in the AutoTest environment. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add a **Script** block and paste the following: ```bash cg junit5 install ``` {% hint style="info" %} `cg junit5 install` downloads the JUnit 5 platform into the AutoTest environment and makes its classpath available via `cg junit5 get-classpath`. This only needs to be run once in Setup — all subsequent test blocks in the same AutoTest configuration can reference it. {% endhint %} ### Step 2: Compile the student code and place the instructor tests 1. Navigate to the **Tests** tab under the AutoTest settings. 2. Add an **Upload Files** block and attach your instructor test file (e.g. `CalculatorTest.java`). 3. Add a **Script** block immediately after and paste the following: ```bash #!/usr/bin/env bash cg java compile $STUDENT/myapp/*.java mv $UPLOADED_FILES/*.java $STUDENT/myapp ```

An Upload Files block with the teacher’s test file, followed by a Compilation script.

{% hint style="info" %} `cg java compile` compiles the student's Java sources and attaches any compiler errors as inline comments on their submission. Moving the uploaded test into `$STUDENT/myapp` places it alongside the student's classes inside the same package. {% endhint %} ### Step 3: Run the JUnit 5 tests 1. Add a **Connect Rubric** block and select the appropriate rubric category from the drop-down menu. 2. Add a **Custom Test** block and nest it within the **Connect Rubric** block. 3. Paste the following into the editor: ```bash # Compile the test file javac -cp "/home/codegrade/student:/home/codegrade/student/myapp:`cg junit5 get-classpath`" \ myapp/CalculatorTest.java # Run the JUnit tests and write the report java -cp "/home/codegrade/student:/home/codegrade/student/myapp:`cg junit5 get-classpath`" \ org.junit.platform.console.ConsoleLauncher \ --select-class myapp.CalculatorTest \ --fail-if-no-tests \ --reports-dir '/tmp/reports' # Gather the results and score cg junitxml /tmp/reports/*.xml ```

A Connect Rubric containing a Custom Test block that compiles and runs the teacher’s tests.

{% hint style="info" %} `cg junit5 get-classpath` returns the classpath for the JUnit 5 platform installed in Step 1. `cg junitxml` parses the generated JUnit XML report and awards rubric points based on the pass/fail ratio. {% endhint %} {% hint style="warning" %} If your students organise their work across multiple packages, adjust the classpath entries and the `--select-class` argument accordingly — for example, `myapp.utils.StringUtilsTest`. {% endhint %} ### Conclusion You have set up an automatically graded assignment that handles package-structured student code. Explore more details on [JUnit5 in CodeGrade](https://help.codegrade.com/automatic-grading-guides/java/advanced-java-autograding#junit5) or on running [Checkstyle](https://help.codegrade.com/automatic-grading-guides/java/advanced-java-autograding#checkstyle) alongside your tests. For any questions about custom setups, reach out to our support team at . # Grading test coverage with JaCoCo Writing good unit tests is a skill students need to develop, which means you need a way to grade the tests they write — not just whether they pass, but how thoroughly they exercise the code. JaCoCo is a Java code coverage library that measures what percentage of a reference implementation is exercised by a set of unit tests. The higher the coverage, the more of the reference code the student's tests are touching. This guide walks through uploading a reference implementation alongside the JaCoCo jars, running the student's tests under the JaCoCo agent, and converting the resulting coverage data into a numeric grade. {% hint style="info" %} This guide covers a scenario where **students write the tests** and you grade them against a reference implementation you provide. For the opposite scenario — instructor-written tests grading student code — see [JUnit5](/automatic-grading-guides/java/grading-with-junit5-and-checkstyle) guide. {% endhint %} ### Before you start You will need the following files ready to upload: * `jacocoagent.jar` — the JaCoCo Java agent. Attached to the JVM with `-javaagent`, it instruments classes at runtime and records which lines are executed. * `jacococli.jar` — the JaCoCo command-line tool. It converts the raw coverage data into an XML report that can be graded. * `cov_grade.sh` — a shell script that reads the XML report and prints a numeric grade for the rubric. See the [Appendix](#appendix-cov_grade.sh) for the full script. * Your reference implementation (e.g. `Calculator.java`). Both JaCoCo jars can be downloaded from the [official JaCoCo release page](https://www.jacoco.org/jacoco/). ### Step 1: Install the JUnit console launcher JaCoCo instruments the JVM directly, so the standard `cg junit5` helpers cannot be used here. Instead, download the JUnit5 platform console launcher jar from Maven Central during Setup. 1. Navigate to the **Setup** tab under the AutoTest settings and install JUnit5.

AutoTest Setup to install JUnit5 using Script block

2. Navigate to the **Test** tab and add an **Allow Internet** block. 3. Add a **Script** block inside the **Allow Internet** block and paste the following: {% code overflow="wrap" %} ```bash curl -O https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.10.0/junit-platform-console-standalone-1.10.0.jar mv junit-platform-console-standalone-1.10.0.jar junit-platform-console-standalone.jar ``` {% endcode %}

An Allow Internet block wrapping a Script that downloads the JUnit5 console launcher.

{% hint style="info" %} The **Allow Internet** block is required in **Test** because `curl` needs to reach Maven Central. Renaming the jar drops the version number so the rest of your scripts can reference a stable path. {% endhint %} ### Step 2: Compile the student's tests against a reference implementation 1. In **Tests** tab, add an **Upload Files** block and attach `jacocoagent.jar`, `jacococli.jar`, `cov_grade.sh`, and your reference implementation (e.g. `Calculator.java`). 2. Add a **Connect Rubric** block and select the appropriate rubric category from the drop-down menu. 3. Add a **Script** block nested inside the **Connect Rubric** block and paste the following: ```bash mv $UPLOADED_FILES/* . javac -cp ".:$STUDENT:junit-platform-console-standalone.jar" \ Calculator.java \ $STUDENT/CalculatorTest.java ```

A Connect Rubric containing the Compilation script.

{% hint style="info" %} Moving the uploaded files into the working directory first ensures that the compiler and the JaCoCo jars can all find each other on the classpath. {% endhint %} ### Step 3: Run the tests under JaCoCo and grade the coverage This step uses two **Custom Test** blocks nested inside the same **Connect Rubric** block from Step 2. #### Custom Test 1 — Run student unit tests Add a **Custom Test** block and paste the following: ```bash # Run tests with JaCoCo agent, output to JUnit XML java -javaagent:./jacocoagent.jar \ -cp ".:$STUDENT:junit-platform-console-standalone.jar" \ org.junit.platform.console.ConsoleLauncher \ execute \ --disable-banner \ --details=none \ --reports-dir=junitxml \ --select-class=CalculatorTest cg unit-tests junitxml junitxml/*.xml ``` {% hint style="info" %} The `-javaagent` flag attaches the JaCoCo agent to the JVM, which instruments the reference implementation's classes at runtime and writes the raw coverage data to `jacoco.exec`. `cg unit-tests` parses the JUnit XML output so pass/fail information appears in the submission view. {% endhint %} #### Custom Test 2 — Grade code coverage Add a second **Custom Test** block immediately after and paste the following: ```bash # Directory layout required by JaCoCo's report command mkdir src tests compiled mv Calculator.java src mv Calculator.class compiled mv $STUDENT/CalculatorTest.class tests rm $STUDENT/CalculatorTest.java # Generate the XML coverage report java -jar jacococli.jar report jacoco.exec \ --classfiles compiled \ --sourcefiles src \ --xml cov.xml # Grade based on the XML report ./cov_grade.sh cov.xml ```

The Custom Test blocks for running student unit tests and grading code coverage.

{% hint style="warning" %} The directory structure (`src`, `tests`, `compiled`) is required by JaCoCo's `report` command. Skipping or renaming these folders will cause the report step to fail. {% endhint %} ### Conclusion You have set up an automatically graded assignment that measures how thoroughly students test a reference implementation. For background on standard instructor-written JUnit 5 tests, or for adding Checkstyle to your Java assignments, see the guides below. For questions about custom grading scripts or coverage thresholds, reach out to our support team at .
Appendix: cov_grade.sh The script below parses the JaCoCo XML report and outputs a line coverage grade as a fraction (e.g. `3/4`). CodeGrade uses this fraction to award rubric points proportionally. If LINE coverage data is not present in the report, the script falls back to INSTRUCTION coverage. If the XML cannot be parsed at all, it returns a grade of `0/1`. ```bash #!/usr/bin/env bash set -uo pipefail COV_XML="${1:-cov.xml}" # If FD 3 is open, write structured JSON there. Otherwise skip JSON output. if { true >&3; } 2>/dev/null; then JSON_FD=3 else JSON_FD="" fi python3 - "$COV_XML" "${JSON_FD:-}" < 2 else "" def emit_json(obj) -> None: if not json_fd: return data = (json.dumps(obj) + "\n").encode("utf-8") os.write(int(json_fd), data) try: root = ET.parse(path).getroot() except Exception: covered = 0 missed = 1 total = 1 frac = Fraction(0, 1) print("Code Coverage Grade") print("-----------------------") print(f"Covered={covered}") print(f"Missed={missed}") print(f"Total={total}") print(f"Grade={frac.numerator}/{frac.denominator}") emit_json({ "tag": "points", "points": f"{frac.numerator}/{frac.denominator}" }) sys.exit(0) covered = None missed = None for counter in root.findall("counter"): if counter.get("type") == "LINE": missed = int(counter.get("missed", "0")) covered = int(counter.get("covered", "0")) break if covered is None: for counter in root.findall("counter"): if counter.get("type") == "INSTRUCTION": missed = int(counter.get("missed", "0")) covered = int(counter.get("covered", "0")) break if covered is None: covered = 0 missed = 1 total = 1 frac = Fraction(0, 1) else: total = covered + missed frac = Fraction(covered, total) if total > 0 else Fraction(0, 1) print("Code Coverage Grade") print("-----------------------") print(f"Covered={covered}") print(f"Missed={missed}") print(f"Total={total}") print(f"Grade={frac.numerator}/{frac.denominator}") emit_json({ "tag": "points", "points": f"{frac.numerator}/{frac.denominator}" }) PY exit 0 ``` {% hint style="info" %} The script expresses the grade as an exact fraction (e.g. `7/10`) rather than a percentage. CodeGrade reads this fraction directly to calculate rubric points, so the output format must not be changed. When `LINE` coverage data is absent from the report — which can happen with certain JVM configurations — the script automatically falls back to `INSTRUCTION` coverage. Both metrics are present in every standard JaCoCo XML report. If the XML file cannot be parsed (for example because the JaCoCo agent failed to write `jacoco.exec`), the script returns `0/1` rather than exiting with an error, so the AutoTest run completes and the student receives a zero for the coverage category instead of a broken result. {% endhint %}
# Jupyter Notebook This guide explains how to configure automatic grading for Jupyter Notebook submissions using CodeGrade’s AutoTest v2 framework. ## 1. Submission Settings To allow students to submit `.ipynb` files, ensure that the **Editor** is enabled in the **Submission Settings** of your assignment.\ You do not need to upload a template file at this stage.
## 2. Building the AutoTest ### Setup Phase In the Setup phase: * Install any required dependencies (e.g., `matplotlib`, `numpy`, `pandas`) using `pip`. * Upload any additional files needed during testing.
### Test Phase In the Test phase, begin by adding a **Jupyter Notebook** block.
#### Uploading a template file Within the block, upload your template notebook. This will serve as the reference structure for testing, as well as a starting file for your students.
To generate a compatible template notebook with CodeGrade cell tagging: 1. Click on the uploaded notebook in the block to preview it.
2. In the pop-up window, download the file—this version contains the necessary cell tags for referencing individual cells in test blocks. You can now fill in the answers in this downloaded notebook using your preferred editor. This filled in file will serve as the **Test Submission**. #### Creating a template file from scratch If you’d like to create a new full template file from within CodeGrade you can: 1. Click on "**Create Empty**" in the **Jupyter Notebook** block 2. Upload a test submission (it doesn’t have to be the actual test submission you will use later) and publish the snapshot 3. Enter Student View and create a new submission
4. You can build your Notebook from within the editor by adding and ordering cells. Submit this file and download it.
## 3. Internet Access in Jupyter Notebooks By default, internet access is disabled whenever notebook code runs, both while your AutoTest tests run *and* while students run their own cells in the notebook editor. Internet is only enabled during the AutoTest Setup phase; after that, all notebook code executes without connectivity. This means code that downloads a dataset, calls an external API, or runs `pip install` from within the notebook will fail during testing *and* when a student runs it in the editor — unless you explicitly allow it. To grant internet access, wrap the **Jupyter Notebook** block inside an **Allow Internet** block: 1. In the Test phase, add an **Allow Internet** block. 2. Nest your **Jupyter Notebook** block (including its **Run In Notebook** blocks) inside it.
All blocks nested inside the *Allow Internet* block will run with internet access enabled. This applies to both AutoTest runs and student cell execution in the editor. {% hint style="info" %} Where possible, install dependencies and prepare environment in the **Setup phase** instead of at test time. This keeps test runs faster and more reliable.\ \ For more information on the *Allow Internet* block, visit [AutoTest V2 Blocks](https://help.codegrade.com/automatic-grading-guides/autotest-v2-blocks). {% endhint %} ## 4. Inserting Test Code To test specific cells in the student’s notebook, nest **Run In Notebook** blocks within the **Jupyter Notebook** block. You can insert test code: * At the start of the notebook * At the end of the notebook * Before or after a specific cell In most use cases, inserting code after each question cell is ideal. For example, you may want to test the function a student writes with simple assertions. To do this: 1. Choose “**After a cell**” from the insertion options.
2. Click on the "**Cell**" field to open the notebook preview.
3. Select the cell after which you want the test code inserted. Write your testing code in the **Run In Notebook** block.
4. Repeat this process for each question or notebook cell you wish to test.
## 5. Publishing the snapshot With your Notebook blocks now set up, click the bottom right **Build Snapshot** button to build your AutoTest Snapshot. The first time you build a snapshot, you are required to upload a Test Submission; in this case it will be built for you from the template notebook you created. Simply click to generate a Test Submission:
## Example: Testing plots You can even test plots programmatically. Below is an example of assertions for a simple plot of `y = x^2:` ```python assert list(line.get_xdata()) == x, "X data does not match expected values" assert list(line.get_ydata()) == y, "Y data does not match expected values" assert ax.get_xlabel() == 'x', "X-axis label incorrect" assert ax.get_ylabel() == 'y', "Y-axis label incorrect" assert ax.get_title() == 'y = x^2', "Plot title incorrect" ``` These assertions verify that the correct data was plotted and the axis labels and title match expectations. # MySQL In this tutorial we will guide you through all the steps to create your very first MySQL assignment in CodeGrade. For this tutorial we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/eE3l7dywPJO2krvRvR9m" %} **A quick note about this example**\ In this assignment, students have been asked to create 3 MySQL queries. Each query should make a selection from the database of a record store: * customers\_from\_canada.sql - The customer ID, last name, first name, and company of each customer based in Canada * songs\_from\_hendrix.sql - the name of every song sold by the store that was written by Jimi Hendrix * top\_5\_tracks.sql - The name and number of purchases of the five most popular tracks sold by the store. ## Step 1: Create an assignment Firstly, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** page by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.

Submission settings with the File uploader and Editor options enabled and a template file uploaded.

3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip MySQL.zip and upload all the files in the Template folder. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own files. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [Submission Settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric category: * **Discrete category** - Specify points in discrete steps (eg. 0, 5 or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 3 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
Customers from CanadaDo you correctly find the CustomerId, FirstName, LastName and Company attributes for all customers from canada?0 - 10
Songs from HendrixDo you correctly find the name of all songs composed by Jimi Hendrix?0 - 10
Top 5 tracksDid you correctly find the name and number of purchases of the top 5 most popular tracks?0 - 10
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial we will be setting up IO tests. ### Step 4.1: Setup
Before actually creating the tests, we need to make sure that we have MySQL installed correctly and that we have prepared the MySQL database and server. 1. Navigate to the Setup tab under the AutoTest settings. 2. Add an Upload Files block to your configuration and upload the chinook.sql file from MySQL.zip. 3. Add a Script block to your configuration and enter the following commands:
```sh sudo apt update sudo apt install mysql-server mysql-client sudo mysql < $UPLOADED_FILES/chinook.sql cat <

A graded IO test that is only run when a submission is made.

Input/Output tests, or "IO tests" for short, are a great way of checking a program's functionality by providing various input cases that should result in a different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Add three **Connect rubric** blocks to your AutoTest configuration. 2. Add three **IO test** blocks and nest one within each Connect rubric block. 3. In each IO test block, provide the following command to run the MySQL query, making sure to adjust the filename appropriately:
```sh MYSQL_PWD='password' mysql --database="Chinook" -t "$@" < customers_from_canada.sql ``` 4. Add a **Full match** block to each IO Test block. 5. Copy and paste the text below into the Expected output field in their respective match block:
customers_from_canadasongs_from_hendrixtop_5_tracks

+------------+-----------+----------+---------------+------------------------+-------------+-------+---------+------------+-------------------+-------------------+------------------------+--------------+
| CustomerId | FirstName | LastName | Company       | Address                | City        | State | Country | PostalCode | Phone             | Fax               | Email                  | SupportRepId |
+------------+-----------+----------+---------------+------------------------+-------------+-------+---------+------------+-------------------+-------------------+------------------------+--------------+
|          3 | François  | Tremblay | NULL          | 1498 rue Bélanger      | Montréal    | QC    | Canada  | H2G 1A7    | +1 (514) 721-4711 | NULL              | ftremblay@gmail.com    |            3 |
|         14 | Mark      | Philips  | Telus         | 8210 111 ST NW         | Edmonton    | AB    | Canada  | T6G 2C7    | +1 (780) 434-4554 | +1 (780) 434-5565 | mphilips12@shaw.ca     |            5 |
|         15 | Jennifer  | Peterson | Rogers Canada | 700 W Pender Street    | Vancouver   | BC    | Canada  | V6C 1G8    | +1 (604) 688-2255 | +1 (604) 688-8756 | jenniferp@rogers.ca    |            3 |
|         29 | Robert    | Brown    | NULL          | 796 Dundas Street West | Toronto     | ON    | Canada  | M6J 1V1    | +1 (416) 363-8888 | NULL              | robbrown@shaw.ca       |            3 |
|         30 | Edward    | Francis  | NULL          | 230 Elgin Street       | Ottawa      | ON    | Canada  | K2P 1L7    | +1 (613) 234-3322 | NULL              | edfrancis@yachoo.ca    |            3 |
|         31 | Martha    | Silk     | NULL          | 194A Chain Lake Drive  | Halifax     | NS    | Canada  | B3S 1C5    | +1 (902) 450-0450 | NULL              | marthasilk@gmail.com   |            5 |
|         32 | Aaron     | Mitchell | NULL          | 696 Osborne Street     | Winnipeg    | MB    | Canada  | R3L 2B9    | +1 (204) 452-6452 | NULL              | aaronmitchell@yahoo.ca |            4 |
|         33 | Ellie     | Sullivan | NULL          | 5112 48 Street         | Yellowknife | NT    | Canada  | X1A 1N6    | +1 (867) 920-2233 | NULL              | ellie.sullivan@shaw.ca |            3 |
+------------+-----------+----------+---------------+------------------------+-------------+-------+---------+------------+-------------------+-------------------+------------------------+--------------+


+--------------------------+
| Name                     |
+--------------------------+
| Foxy Lady                |
| Manic Depression         |
| Red House                |
| Can You See Me           |
| Love Or Confusion        |
| I Don't Live Today       |
| May This Be Love         |
| Fire                     |
| Third Stone From The Sun |
| Remember                 |
| Are You Experienced?     |
| Stone Free               |
| Purple Haze              |
| 51st Anniversary         |
| The Wind Cries Mary      |
| Highway Chile            |
| The Star Spangled Banner |
+--------------------------+


+----------------------+-------+
| Name                 | Count |
+----------------------+-------+
| Dazed and Confused   |     5 |
| The Trooper          |     5 |
| Eruption             |     4 |
| Hallowed Be Thy Name |     4 |
| Sure Know Something  |     4 |
+----------------------+-------+
{% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to both you and your students what is actually being checked (e.g. "Does customers\_from\_canada.sql produce the correct output?"). {% endhint %} ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop all the files in the "Solution" folder from MySQL.zip and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to **Upload Submission** option in the **General** settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit". Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish snapshot** button. ## Conclusion You have just built your first Automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](/faq) or reach out to our support team at . # Web Development ## JavaScript {% content-ref url="/pages/lpOxHO4K5VTU2wLNfJkm" %} [Create your first JavaScript assignment](/automatic-grading-guides/javascript/create-your-first-javascript-assignment) {% endcontent-ref %} In CodeGrade you can automatically test web development assignments. {% content-ref url="/pages/bGVPH4dxeUJ6emgDjjZI" %} [Web Development with Selenium](/automatic-grading-guides/web-development/web-development-with-selenium) {% endcontent-ref %} {% content-ref url="/pages/IlWLgfbLwsIKmDwlK0m3" %} [Automatically grading CSS](/automatic-grading-guides/web-development/automatically-grading-css) {% endcontent-ref %} # Web Development with Selenium ## Overview This guide illustrates how to automatically test web development assignments in CodeGrade using Jest and Selenium. [Selenium](https://www.selenium.dev/) is a powerful open-source testing tool for web development that allows you to test the webpages created by your students functionally. For instance, you may use Selenium to test that webpage page created by the student: * Contains the required `html` elements such as tables, headings, and paragraphs; * Responds to user interaction as expected, for example, by using buttons and inputs. To automatically test the CSS properties of the student's page, see the dedicated guide [here](/automatic-grading-guides/web-development/automatically-grading-css). Below, we are going to set up AutoTest to test the student's implementation of the Simon game. The student submission includes: * a `html` file named `index.html;` * a javascript file named`game.js` that implements the webpage functionalities; * the required `jquery` library in its minified version; * a folder `sounds` that contains the `mp3` files to play when buttons are pressed; * optionally, a `css` file that defines `css` rules to enhance the webpage's appearance. This is not a necessary file here, as we won't be grading `css` rules in this guide. Below, you can download the submission folder for this assignment: {% file src="/files/LwS3VjKi4xaV8bwCwWFV" %}

Example webpage submission rendered in CodeGrade

{% hint style="info" %} `html` files can be fully rendered within CodeGrade. This means that the students can see how their web pages are displayed and react right within their submission folder. {% endhint %} ## AutoTest Setup In the **setup** section of your AutoTest, you can install software or packages and upload any files you might need for testing. The **setup** section will build an image and is only run once for all submissions. ### Step 1 First, we need to install **Node.js :** drag the ["Install Node" block](https://help.codegrade.com/automatic-grading-guides/javascript#installing-node) to the **setup** step and select the preferred version in the dropdown. ### Step 2 The second step is to install the additional software we need using a [Script Block](about:blank): {% code lineNumbers="true" fullWidth="false" %} ```bash # Update Ubuntu packages sudo apt update # Install Google Chrome wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chrome-linux64.zip &> /dev/null unzip chrome-linux64.zip &> /dev/null rm -f ./chrome-linux64.zip chmod +x ./chrome-linux64/chrome sudo ln -s ${PWD}/chrome-linux64/chrome /usr/local/bin/chrome # Install Chrome dependencies sudo apt install libxkbcommon0 libgbm-dev -y # Install Chromedriver wget https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5845.96/linux64/chromedriver-linux64.zip &> /dev/null unzip chromedriver-linux64.zip &> /dev/null rm -f chromedriver-linux64.zip chmod +x ./chromedriver-linux64/chromedriver sudo ln -s ${PWD}/chromedriver-linux64/chromedriver /usr/local/bin/chromedriver # Install a headless display server sudo apt install xvfb -y # Install Jest npm install -g jest eslint eslint-config-standard eslint-plugin-promise eslint-plugin-import eslint-plugin-n eslint-detailed-reporter ``` {% endcode %} This script installs the following software: * the Google Chrome web browser and its driver; * `xvfb` , a display server that makes it possible to run a web browser using a virtual display; * [`jest`](https://jestjs.io/), the javascript unit test framework on which we run our selenium tests. ### Step 3 Next, we need to [upload our fixtures](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/upload-files-to-autotest), which in this case are: * The `package.json` file that specifies all the node dependencies of the node project. This is where you have to list all the node packages the student may need for the project and you as a teacher may need to implement your tests. In our example, we just need to install the testing dependencies: 1. `selenium-webdriver` : the node package to run selenium in node.js; 2. `chromedriver`: the node package to run the Google Chrome browser; 3. `jest-junit` : the node package that enables jest to write the unit test results in a `xml` format. {% code fullWidth="false" %} ```json { "name": "game", "version": "1.0.0", "description": "test for Simon game", "main": "game.js", "scripts": { "test": "jest" }, "author": "codegrade", "license": "ISC", "dependencies": { "selenium-webdriver":"", "chromedriver":"", "jest-junit":"" } } ``` {% endcode %} * The `game.test.js` file that contains the jest unit tests that use selenium to functionally test the student webpage.
game.test.js Below you have the testing file `game.test.js:` {% code lineNumbers="true" fullWidth="true" %} ```javascript const {Builder, By, Key, until} = require('selenium-webdriver'); require('chromedriver') // This is the directory where the student's submission ends up. // It can easily be changed to any URL. const baseURL = "file://" + __dirname + "/"; // Change this to select another file to test. const file = 'index.html' const defaultTimeout = 1000; // Four simple functions to ease DOM navigation const getElementByName = async (driver, name, timeout = defaultTimeout) => { const element = await driver.wait(until.elementLocated(By.name(name)), timeout); return await driver.wait(until.elementIsVisible(element), timeout); }; const getElementById = async (driver, id, timeout = defaultTimeout) => { const element = await driver.wait(until.elementLocated(By.id(id)), timeout); return await driver.wait(until.elementIsVisible(element), timeout); }; const getElementByTag = async (driver, tag, timeout = defaultTimeout) => { const element = await driver.wait(until.elementLocated(By.tagName(tag)), timeout); return await driver.wait(until.elementIsVisible(element), timeout); }; function sleep(time) { return new Promise(resolve => setTimeout(() => resolve(), time)); } // This is run before any test. beforeAll(async () => { driver = await new Builder().forBrowser('chrome').build(); await driver.get(baseURL + file); }); // This makes sure the browser session exits. afterAll(async () => { await driver.quit(); }); describe('Check setup.', () => { test('Check for title.', async () => { const title = await getElementById(driver, 'level-title'); var title_attr = await title.getProperty('innerHTML'); expect(title_attr).toEqual("Press A Key to Start"); }); test('Check for buttons.', async () => { const greenButton = await getElementById(driver, 'green'); const blueButton = await getElementById(driver, 'blue'); const yellowButton = await getElementById(driver, 'yellow'); const redButton = await getElementById(driver, 'red'); expect(greenButton).not.toBeUndefined(); expect(redButton).not.toBeUndefined(); expect(blueButton).not.toBeUndefined(); expect(yellowButton).not.toBeUndefined(); }); }); describe('Check initialization.', () => { test('Check for game start.', async () => { const body = await driver.findElement(By.css('body')); await body.sendKeys('RETURN'); await sleep(2000); let title = await getElementById(driver, 'level-title'); let title_attr = await title.getProperty('innerHTML'); expect(title_attr).toEqual('Level 1'); }); }); describe('Check Game Mechanics.', () => { test('Check correct press.', async () => { const gamePattern = await driver.executeScript('return gamePattern;'); const buttonOne = await getElementById(driver, gamePattern[0]); await buttonOne.click(); await sleep(2000); let title = await getElementById(driver, 'level-title'); let title_attr = await title.getProperty('innerHTML'); expect(title_attr).toEqual('Level 2'); }); test('Check another correct press.', async () => { const gamePattern = await driver.executeScript('return gamePattern;'); const buttonOne = await getElementById(driver, gamePattern[0]); const buttonTwo = await getElementById(driver, gamePattern[1]); await buttonOne.click(); await buttonTwo.click(); await sleep(2000); let title = await getElementById(driver, 'level-title'); let title_attr = await title.getProperty('innerHTML'); expect(title_attr).toEqual('Level 3'); }); test('Check wrong press.', async () => { const gamePattern = await driver.executeScript('return gamePattern;'); let buttonOne = null; if (gamePattern[0] != 'green') { buttonOne = await getElementById(driver, 'green'); } else { buttonOne = await getElementById(driver, 'red'); } await buttonOne.click(); await sleep(2000); var title = await getElementById(driver, 'level-title'); var title_attr = await title.getProperty('innerHTML'); expect(title_attr).toEqual('Game Over, Press Any Key to Restart'); }); }, defaultTimeout); ``` {% endcode %} The `game.test.js` file contains a first setup part and a second part where we define the actual tests. #### Setup * In lines 2 and 3 the `selenium-webdriver` and `chromedriver` packages are imported; * In line 7 the constant `baseURL` points to the directory where the testing file is found; * In line 11`file` is the name of the student's html file that we will open when starting our headless web browser. Here, we assume that the student's file and the `game.test.js` file was moved to the student directory so that testing and the tested files are in the same directory. * From line 16 to line 29 the functions `getElementbyName`, `getElementById` and `getElementByTag` are defined. These functions help us navigate the web pages programmatically; * In line 31 a function `sleep` is defined. It simply awaits a given number of seconds; * From line 36 to line 44 the `beforeAll` and `afterAll` functions are defined. These are special functions that jest uses for each test setup and teardown. In the setup, we create a new web driver object that points to the student's initial webpage, and in the teardown, we make sure to properly exit the browser session. #### Tests Let's now examine some actual tests. In line 49 we define the test `Check for title.` . This test checks that the `html` page created by the student contains a specific title element. Specifically: * In line 50 we search for an HTML element whose id is `level-title`and save it into a constant named `title`; * In line 51 we access the `innerHTML` property that `title` is expected to have; * In line 52 we check that the string used for the title is as expected. In line 55 we define the test `Check for buttons.` . This test checks that the page contains elements corresponding to specific `html ids`. For example: * line 56 searches for an `html` element with the id `green` and saves it into a constant named `greenButton`; * ln line 60 we check that the `greenButton` variable is properly defined.
### Step 4 Assuming that all the students are going to use the same set of node packages indicated in the previously uploaded `package.json` file, it is convenient to install the node packages during the AutoTest S**etup**.\ This saves time for the students as the same installation won't have to be executed each time a new submission is run. For this, simply drag and drop another Script Block and type: ```bash # Move to the directory where the package.json file is cd $UPLOADED_FILES # Install the node packages indicated in the package.json file npm install ``` The `npm install` creates a folder named `node_modules` inside the fixture folder. Later we can move this folder into the student directory where we run the tests.
## AutoTest Tests ### Step 1 Before running the tests we need to move all the necessary fixtures into the [student directory](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/autotest-v2-filesystem).\ This should be done at the top of the **Tests** phase using a Script Block that runs the following command: ```bash mv $UPLOADED_FILES/node_modules $UPLOADED_FILES/package.json $UPLOADED_FILES/game.test.js . ``` ### Step 2 We can finally run the selenium tests. To do so , drag and drop a[ Custom Test Block](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/create-a-custom-test) and run the following commands: {% code lineNumbers="true" %} ```bash # Run the unit tests and create a report file xvfb-run jest --reporters=jest-junit --testTimeout=1000 # Parse the test report file to display results and compute the grade cg unit-tests junitxml junit.xml ``` {% endcode %} * In line 2 we run `jest`, the javascript unit testing framework, prepending it with `xvfb-run`. This is needed as our selenium tests require a display to open the web browser; `xvfb-run` allows to use a virtual display. The option `--reporters=jest-junit` makes such that `jest` creates an xml report file containing the results of the tests. The option `--testTimeout=10000` indicates a 10 second timeout for each tests. * In line 5 we use the CodeGrade builtin command [`cg`](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/run-and-display-unit-tests/cg-command-for-displaying-unit-test-output) to parse the xml report file produced by `jest`. This visually displays the test results and computes a grade if required. Optionally, you can: * connect the Custom Test Block to a rubric using a [Connect Rubric Block](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/make-your-tests-graded); * [hide information ](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/hide-test-steps)to the students about the run Block, such as its configuration and output.
# Automatically grading CSS In this page we describe how to automatically test css rules for web development assignments. While in principle, it is possible to use [Selenium](https://app.gitbook.com/o/-MKoTuXczViPzaai1tw3/s/-MKAQsDlg_P20iQy3JDs/~/changes/190/faq/automatic-grading-questions/autotest-v2/web-development-assignments-selenium-tests) to test the `css` properties defined for `html` elements, we advise against doing so in CodeGrade. The reason is that we run Selenium tests by using a web browser in headless mode, which can result in `css` rules working differently than they would while running your web browser with a normal display.\ \ Alternatively, we can use [Semgrep](https://semgrep.dev/) to test the structure of the `css` rules defined by the student. ## Design Semgrep patterns for testing CSS rules Semgrep is used to perform code analysis by writing syntax rules in a human-readable format. Rules are made by generic or language-specific patterns, which are then searched for in the student's code. In Semgrep's [pattern syntax](https://semgrep.dev/docs/writing-rules/pattern-syntax/), you have: * **Equivalences**: Matching code that *means the same thing* even though it looks different; * **Wildcards / ellipsis (...)**: Matching any statement, expression or variable; * **Metavariables ($X)**: Matching unknown expressions that you do not yet know what they will exactly look like, but want to be the same variable in multiple parts of your pattern; * **Logical operators:** it is possible to logically combine simpler patterns to create more complex aggregate patterns. Let's inspect an example of a **Semgrep** pattern that checks the properties of a `btn` `css` class: 
patterns:
      - pattern-inside: |
         .btn { ... }
      - pattern: |
         ...
         margin: 20px;
         ...
      - pattern: |
          ...
          width: $ANYVALUE;
          ...
* Line 1: with `patterns` we indicate that our overall pattern is composed of multiple sub-patterns; * Lines 2 to 3: we require that the patterns that will be defined later have to be found within the body of a class named `btn`; * Lines 4 to 7: we require that our pattern contains a margin setting of exactly 20 pixels; * Lines 8 to 11: we require that our pattern contains a width setting with any possible value (notice the use of a metavariable) {% hint style="info" %} We suggest using the online [Semgrep editor](https://semgrep.dev/playground/new) to quickly develop new rules for testing `css` rules. {% endhint %} ## Instructions

A Semgrep test for Css rules. The test configuration is hidden to the students and the test is connected to a rubric category.

1. In the **Setup** section of your AutoTest, drag and drop a **Script Block** into your configuration and type the following bash commands: ```bash python -mvenv ~/.venv/semgrep_1.43.0 source ~/.venv/semgrep_1.43.0/bin/activate python -m pip install semgrep==1.43.0 ``` 2. In the Tests section of your AutoTest, drag and drop a **Code Structure Test** block. Specify the name of the CSS file you want to test in the "Student file" field. 3. Drag and drop a **Positive Match** block into your configuration and nest it within the Code Structure Test block. Copy and paste your Semgrep rule in the provided editor. 4. Build and Publish your snapshot. # JavaScript ## Create your first autograded JavaScript assignment {% content-ref url="/pages/lpOxHO4K5VTU2wLNfJkm" %} [Create your first JavaScript assignment](/automatic-grading-guides/javascript/create-your-first-javascript-assignment) {% endcontent-ref %} ## Advanced grading options for JavaScript assignments {% content-ref url="/pages/ceWoC8b3zvS6u82rolUy" %} [Advanced JavaScript autograding](/automatic-grading-guides/javascript/advanced-javascript-autograding) {% endcontent-ref %} # Create your first JavaScript assignment This tutorial will guide you through all the steps to create your first JavaScript assignment in CodeGrade. For this tutorial, we will consider the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/wXWcqucfvvjoxIRzw2Af" %} **A quick note about this example**\ In this assignment, students have been asked to create a JavaScript program that prompts the user for their name in the console and prints a greeting. It is a purely console-based application. ## Step 1: Create an assignment First, we need an assignment to work with. Whether CodeGrade is integrated into your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the " :gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the General tab.

Submission settings with the File uploader and Editor options enabled and a template file uploaded.

3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Greeting.zip folder and upload Greeting/Template/greeting.js file as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [submission settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and clarify the grading requirements for your students. CodeGrade offers two types of rubric categories: * **Discrete category** - Specify points in discrete steps (e.g. 0, 5, or 10 points) * **Continuous category** - Specify points as a continuous scale (e.g. 0 - 10) 1. Navigate to the **Rubric** tab on the Assignment Management page. 2. Select **Create new rubric**. 3. Create 2 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
IO testsCheck that your program produces the expected output given a specific input.0 - 80
Code structure testsCheck that your program uses the required code structures.0 - 20
## Step 4: Create automatic tests Automatic tests are really what allow your students to learn uniquely. The immediate feedback they get from the automatic tests allows them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial, we will set up two automatic tests: IO tests and Semgrep code structure tests. ### Step 4.1: Setup

An Install Node block in the Setup of AutoTest v2

Before creating the tests, we need to ensure we have the correct version of Node installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add an **Install Node** block to your configuration. 3. Select Node version 18 to install. ### Step 4.2: Input/Output tests
Input/Output tests (or "IO tests" for short) are a great way of checking a program's functionality by providing various input cases that should result in different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Navigate to the **Tests** tab of the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration. Select the "IO tests" rubric category from the drop-down menu. 3. Add an **IO test** block to your AutoTest configuration and nest it inside the Connect Rubric block. 4. In the editor provided, use the command `node greeting.js` to run the program. 5. Add three **Substring match** blocks to your AutoTest configuration and nest them inside the IO test block. Make sure each match block is case insensitive and ignores whitespaces. 6. Copy and paste the info from the table below into the **Input** and **Expected output** fields in their respective match block: | Input | Expected output | | ----------------------- | --------------- | |

James

| Hello, James! | |

Alicia

| Hello, Alicia! | |

{empty line}

| Invalid | {% hint style="danger" %} Each input must have a trailing newline for it to be recognized (i.e., "James\n"). {% endhint %} {% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to both you and your students what is actually being checked (e.g., "Does your program produce the correct output? - Input: 5"). {% endhint %} ### Step 4.3: Code structure tests

A graded Code structure test in AutoTest v2

Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax to reach their solution. AutoTest v2 offers the **Code structure test** block and its corresponding **Positive match** and **Negative match** blocks for running Semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example, we want to check that the students have used an if-else statement in their program. Here is the Semgrep rule we will be using: {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: if (...) {...} else {...} message: Semgrep found a match languages: [javascript] severity: WARNING ``` {% endcode %} 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "greeting.js". 4. Add a **Positive match** block to your test configuration and nest it within the Code structure test block. 5. Copy and paste the Semgrep rule in the code block above into the provided editor. Make sure to delete the template provided. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to ensure your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. CodeGrade will prompt you to upload a Test submission the first time you build a snapshot. Use the "Click here to upload files" option or drag-and-drop the Greeting/Solution/greeting.js file you downloaded at this tutorial's beginning and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit". Once you're happy with your tests, you must publish them to your students so they can run on their submissions. Publish your tests by building a snapshot, and at the bottom of the pop-up modal, click the **Publish to students** button. Once it is published, AutoTest V2 is available for students through the editor and on the submission page. ## Conclusion You have just built your first automatically graded assignment and are ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade offers. For more in-depth information about the product and the various workflows you can achieve, see [Learn more](/faq) or contact our support team at . # Advanced JavaScript autograding Discover the advanced grading options available for JavaScript assignments In this guide, we explore the advanced grading options available for JavaScript assignments. For more information about setting up a JavaScript assignment from scratch, see: {% content-ref url="/pages/lpOxHO4K5VTU2wLNfJkm" %} [Create your first JavaScript assignment](/automatic-grading-guides/javascript/create-your-first-javascript-assignment) {% endcontent-ref %} ## Jest Jest is an industry-standard unit testing framework for JavaScript. It is great for grading assignments in which students create their own functions. Consider the example submission below: {% code title="calculator.js" %} ```javascript // Function to add two numbers function add(a, b) { return a + b; } // Function to subtract the second number from the first function subtract(a, b) { return a - b; } // Function to multiply two numbers function multiply(a, b) { return a * b; } // Function to divide the first number by the second // Throws an exception if the second number is 0 function divide(a, b) { if (b === 0) { throw new Error("Division by zero is not allowed."); } return a / b; } // Export the functions for use in other modules module.exports = { add, subtract, multiply, divide }; ``` {% endcode %} Using Jest, we can easily create multiple test cases to assess the functionality of the functions. {% code title="calculator.test.js" %} ```javascript const { add, subtract, multiply, divide } = require('./calculator'); describe('Calculator Functions', () => { test('add function adds two numbers', () => { expect(add(1, 2)).toBe(3); expect(add(-1, -1)).toBe(-2); expect(add(1.5, 2.5)).toBe(4); }); test('subtract function subtracts the second number from the first', () => { expect(subtract(5, 3)).toBe(2); expect(subtract(0, 5)).toBe(-5); expect(subtract(5.5, 2.5)).toBe(3); }); test('multiply function multiplies two numbers', () => { expect(multiply(2, 3)).toBe(6); expect(multiply(-2, -3)).toBe(6); expect(multiply(2.5, 2)).toBe(5); }); test('divide function divides the first number by the second', () => { expect(divide(6, 3)).toBe(2); expect(divide(-6, -3)).toBe(2); expect(divide(5.5, 2)).toBe(2.75); }); test('divide function throws an error when dividing by zero', () => { expect(() => divide(1, 0)).toThrow('Division by zero is not allowed.'); }); }); ``` {% endcode %} ### "cg junitxml" command We could simply compile and run the unit test to show students a simple pass or fail, depending on the exit code of the tests. However, we wouldn't be able to award partial marks for each test case unless we parsed the results of the unit tests. To accomplish that, we can use the `cg junitxml` command. This command allows us to parse the number of test cases passed and the feedback from failed test cases from any unit test report written in the JUnit XML format. Running the command `cg junitxml --help` shows the following information: ``` This is a parser for generic unit test coverage reports in JUnit XML format. The parser can take in input multiple JUnit XML files. The final reported score is an aggregate of all the reports. Skipped test cases do not count towards the score. Example use: Input to the command (in a file called coverage.xml): ERROR: Expected: 6.0 but was: 5.0 Category: Checking returns - Multiplication File: /home/codegrade/student/calculator.py Line: 2 You would then call: cg junitxml coverage.xml Usage: cg junitxml XMLS... [flags] Flags: -h, --help help for junitxml --no-parse-weight Don't try to parse the weight from titles. --no-score Don't calculate and output the final score based on the results of the input data. -o, --output string The output file to to use. If this is a number it will use that file descriptor. (default "3") ``` The `cg junitxml` command works well in combination with the Custom Test block by displaying the results of the parsed JUnit XML report beautifully, making it easy to read and interpret, as shown in the image below.
Fortunately, we can use the "jest-junit" reporter option to automatically output a JUnit XML report from our unit test. We can set the reporter in our package.json file: {% code title="package.json" %} ```json { "dependencies": { "jest": "^29.7.0", "jest-junit": "^16.0.0" }, "scripts": { "test": "jest" }, "jest": { "reporters": [ "jest-junit" ] } } ``` {% endcode %} ### Instructions
1. In the AutoTest settings, navigate to the **Setup** tab. 2. Add an **Install Node** block and select the latest version. 3. Add an **Upload Files** block. Upload `calculator.test.js` and `package.json`. 4. Add a **Script** block. Use the following bash commands to install the node modules for jest and the jest-junit reporter: 
cd $UPLOADED_FILES
   npm install
5. Navigate to the **Tests** tab. 6. Add a **Script** block. Use the following bash commands to make the unit tests, node modules, and package.json available to the students' submissions: ```bash mv $UPLOADED_FILES/* . ``` 7. Add a **Connect Rubric** block and a **Custom Test** block. Nest the Custom Test block within in the Connect Rubric block. Use the following command to run Jest and parse the resulting JUnit XML report: ```bash npm test calculator.test.js --silent cg junitxml junit.xml ``` 8. Build and publish your snapshot. ## ESLint [ESLint](https://eslint.org) is an industry-standard static code analysis tool for JavaScript that allows you to identify styling and implementation issues in your code. It is a useful tool for enforcing code styling best practices for beginner programmers. ### "cg comments" command Simply running ESLint would not be particularly useful for students as they would have to spend time interpreting the command line output and would have to switch back and forth between the AutoTest output and their code. Instead, we can use the `cg comments` command to parse the output of ESLint and write the comments directly onto our students' code. Running the command `cg comments --help` shows the following information: ``` This is a parser for generic linters to CodeGrade ATv2 comments. The regex should be written in Perl5 flavor and should have at least 3 match groups: file: The filename of the comment. line: The line of the comment, by default this will be the start and endline. message: The message of the comment. There are 5 optional match groups: column: The (begin) column of the error, defaults to 0. lineEnd: The end line of the error, defaults to the begin line. columnEnd: The end column of the error, defaults to the end of the line. code: The error code, any string is accepted. severity: The severity, valid values are "fatal","error","warning", and "info". Will default to not set. If the parsed severity is not valid, it will also default to not set. Example use: Input to the command: filename:1:Some error that has occurred on line 1 different_filename:25:An error on line 25 of a different file Regex to pass to the command to capture filename, line, and message: ^(?P[^:]*):(?P\d+):(?P.*)$ Resulting in the following invocation piping the input to cg comments: input_to_command | cg comments \ '^(?P[^:]*):(?P\d+):(?P.*)$' \ --origin "the-linter" Usage: cg comments REGEX [flags] cg comments [command] Available Commands: generic A generic reporter that uses regex to parse log messages. Flags: --base-path string The base path of the reported files. Files that are not children of this path will not be reported. Pass the empty string to disable this feature. (default "/home/codegrade/student") --buffer-time duration The time comments are buffered to be grouped together into a single message. (default 2s) --deduction-error Deduction The percentage deduction for comments with *error* severity. It should be an integer value. (default 20) --deduction-fatal Deduction The percentage deduction for comments with *fatal* severity. It should be an integer value. (default 100) --deduction-info Deduction The percentage deduction for comments with *info* severity. It should be an integer value. (default 5) --deduction-unknown Deduction The percentage deduction for comments with no set severity. It should be an integer value. (default 0) --deduction-warning Deduction The percentage deduction for comments with *warning* severity. It should be an integer value. (default 10) -h, --help help for comments --ignore-parser-errors Should we ignore it if we encounter a line in the input that we cannot parse using the regex provided. --ignore-regex string Lines matching this regex will be ignored. --no-score If enabled, the score will not be generated based on the comments according to the set deduction for each severity level. To customize these values when disabled, add a --deduction- [percentage] flag for each severity you wish to override. To apply a deduction to comments with no severity set, set the --deduction-unknown value to a number greater than 0. --origin string The origin to use for the linter. -o, --output string The output file to to use. If this is a number it will use that file descriptor. (default "3") -p, --print Print comments after parsing them. --severities string A lookup mapping for parsed severities. Should be of the format: "parsed1:severity,parsed2:severity", for example "note:info,remark:info". Use "cg comments [command] --help" for more information about a command. ``` The `cg comments` command will highlight each target line according to the severity of the comment and can be read by hovering over the line number with the mouse cursor. The comments are also placed on students' code in the editor, making it a powerful combination.
### Requirements ESLint requires a configuration file and a package.json file to be run without errors. Here you can find an example of a `package.json` file and a `eslint.config.js` file. {% code title="package.json" %} ```json { "dependencies": { "eslint": "^9.3.0", "eslint-plugin-jsdoc": "^48.2.6", "eslint-formatter-compact": "^8.40.0" }, "type": "module" } ``` {% endcode %} Here, we require the jsdoc plugin and the compact formatter for eslint. The jsdoc plugin provides extra rules related to code documentation and the compatc formatter allows us to format the output of ESLint. Additionally, we indicate `"type"` as `"module"` so that the config file is treated as a module. {% code title="eslint.config.js" %} ```javascript import jsdoc from "eslint-plugin-jsdoc"; export default [ { files: ["**/*.js"], plugins: { jsdoc: jsdoc }, rules: { "jsdoc/require-description": "error", "jsdoc/check-values": "error", // Indentation "indent": ["warn", 2], // Semicolons "semi": ['warn', 'always'], "semi-style": ['warn', 'last'], // Empty lines "no-multiple-empty-lines": ["warn", { "max": 1, "maxBOF": 0, "maxEOF": 0 }], // Spaces "no-multi-spaces": "warn", "space-infix-ops": ["warn", { "int32Hint": false }], "space-in-parens": ["warn", "never"], "no-trailing-spaces": "warn", "semi-spacing": ["warn", {"before": false, "after": true}], // Variable naming // WARNING: does not check if PascalCase is used instead of camelCase "camelcase": ["warn", { "properties": "always"}], "id-length": ["warn", { "min": 2, 'exceptions': ['i', 'j', 'n', 'm', 'PI', 'Pi', 'pi'] }], "no-underscore-dangle": ["warn"], // Other "no-var": "warn", "prefer-const": "warn", } } ]; ``` {% endcode %} For more information about ESLint rules and configuration, [please see their documentation](https://eslint.org/docs). ### Instructions
1. In the AutoTest settings, navigate to the **Setup** tab. 2. Add an **Install Node** block and select the most recent version of Node to install. 3. Add an **Upload Files** block and upload both your `package.json` file and your `eslint.config.js` file. 4. Add a **Script** block and use the following commands to install eslint and the dependencies listed in the `package.json` file. ```bash cd $UPLOADED_FILES && npm install ``` 5. Navigate to the **Tests** tab. 6. Add a **Connect Rubric** block and a **Custom Test** block. Nest the Custom Test block in the Connect Rubric block. Use the following commands to move the uploaded files and dependencies you installed to the student directory, run ESLint, and parse the result using the `cg comments` command. ```bash mv $UPLOADED_FILES/* . npx eslint --format compact calculator.js | cg comments \ '(?P/[^:]*): line (?P[\d]+), col (?P[\d]+), (?P[\S]+) - (?P.*) \((?P[^\)]*)\)$' \ --origin eslint \ --severities 'Error:error,Warning:warning' \ # Map unknown severities --ignore-regex '^\s*$|^[^/].*$' \ # Ignore this pattern in the ESLint output ``` 7. Build and publish your snapshot ### Conclusion These advanced JavaScript testing options open the door to more rigorous testing methods and better student feedback. However, these are just the most commonly used frameworks used for testing JavaScript. There are many more testing tools and packages available that you can use with some simple setup. For more information, contact us at . # Running Docker ## Grade a Dockerized web app In this guide we explore how to grade a containerized web application with CodeGrade. Students submit a small web app together with a `Dockerfile`; CodeGrade builds the image, runs the container, and checks that the website actually responds over HTTP. The same Setup + Tests pattern works for any Dockerized project — Node, Go, .NET, a database — not just the Flask example used here. Consider the example submission below. It is a tiny [Flask](https://flask.palletsprojects.com/) web server that listens on host `0.0.0.0`, port `8080`, and serves three endpoints: | Method & path | Response | | ------------------ | ------------------- | | `GET /` | `Hello, CodeGrade!` | | `GET /health` | `OK` | | `GET /add//` | the sum of a and b | {% tabs %} {% tab title="app.py" %} ```python from flask import Flask app = Flask(__name__) @app.route("/") def index(): return "Hello, CodeGrade!\n" @app.route("/health") def health(): return "OK\n" @app.route("/add//") def add(a, b): return f"{a + b}\n" if __name__ == "__main__": app.run(host="0.0.0.0", port=8080) ``` {% endtab %} {% tab title="requirements.txt" %} ``` Flask==3.0.3 ``` {% endtab %} {% tab title="Dockerfile" %} ```docker FROM python:3.12-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY app.py . EXPOSE 8080 CMD ["python", "app.py"] ``` {% endtab %} {% endtabs %} > **Note:** This guide drives Docker entirely from Bash **Script** blocks rather than a built-in language block, so it works regardless of the language inside the container. ### AutoTest Setup In the **Setup** phase we install Docker, start the daemon, and open the Docker socket so the (non-root) Tests phase can use Docker without `sudo`. The Setup phase has internet access and `sudo` rights, so this is the right place to do all privileged work. Add a **Script** block to the Setup phase with the following script: ```bash #!/usr/bin/env bash # Install Docker, start the daemon, and open the socket so the (non-root) # test steps can run `docker` WITHOUT sudo. set -euo pipefail export DEBIAN_FRONTEND=noninteractive # 1. Install Docker if it isn't already there. # docker-buildx silences the "legacy builder is deprecated" warning. if ! command -v docker >/dev/null 2>&1; then sudo -n apt-get update sudo -n apt-get install -y --no-install-recommends docker.io docker-buildx || \ sudo -n apt-get install -y --no-install-recommends docker.io fi # 2. Start the daemon (systemd if present, otherwise dockerd directly). if command -v systemctl >/dev/null 2>&1 && sudo -n systemctl start docker 2>/dev/null; then : else sudo -n sh -c 'dockerd >/tmp/dockerd.log 2>&1 &' || true fi # 3. Wait for the daemon socket to come up. for _ in $(seq 1 30); do [ -S /var/run/docker.sock ] && sudo -n docker info >/dev/null 2>&1 && break sleep 1 done # 4. Open the socket so the test step can talk to Docker without sudo. sudo -n chmod 666 /var/run/docker.sock sudo -n docker --version ``` > **Note — why `chmod 666` the socket?** The grading user isn't in the `docker` group, and group membership only takes effect on the next login — so it can't help within a single run. Opening the socket during Setup (where `sudo` works) lets the Tests phase run plain `docker` commands. This is safe for a throwaway grading sandbox. ### AutoTest Tests In the **Tests** phase we upload a small `Test.sh` script, build the student's image, run it, and check that the website responds. Configure the blocks as follows (the names in quotes match the screenshot below): 1. Add an **Allow internet** block as the outermost block. The build needs to pull the base image and install packages, and the Tests phase has no internet by default. 2. Inside it, add an **Upload Files** block ("Upload test") and upload the `Test.sh` script shown below. 3. Also inside Allow internet, add a **Connect rubric** block ("Running the App") and select your rubric category. This ties the pass/fail outcome to the grade. 4. Inside Connect rubric, add an **IO Test** block ("Running docker"). In its command field, copy the uploaded script into the working directory and run it: ```bash # Run the bash script and check the match blocks. cp $FIXTURES/*.* . ./Test.sh ``` 5. Inside the IO Test, add a **Substring Match** block ("Checking the App"). Leave **Input** empty and set **Expected in output** to `Hello, Web app is up!`. Keep it **case-sensitive** and **include whitespaces**.
The Tests phase: an IO Test that runs Test.sh with a Substring Match, nested inside Connect rubric and Allow internet.

The Tests phase: an IO Test that runs Test.sh with a Substring Match, nested inside Connect rubric and Allow internet.

The `Test.sh` script does the actual work: it builds the image, runs the container, and prints `Hello, Web app is up!` when the site responds. The Substring Match looks for exactly that line, so a working submission passes and fills the rubric category. ```bash #!/usr/bin/env bash # Build the student's Dockerfile, run the container, and check the site is up. # Prints the line the Substring Match block looks for. No sudo needed: # the Setup step already opened the Docker socket. set -euo pipefail # Clean up any container left from a previous run. docker rm -f cg-app >/dev/null 2>&1 || true trap 'docker rm -f cg-app >/dev/null 2>&1 || true' EXIT # 1. Build the student's image. docker build -t cg-submission . # 2. Run it in the background. docker run -d --name cg-app cg-submission >/dev/null # 3. Find the container's IP on the Docker network. IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cg-app) # 4. Wait up to 30s for the website to start responding. for _ in $(seq 1 30); do curl -sf "http://${IP}:8080/" >/dev/null 2>&1 && break sleep 1 done # 5. Print the marker when the site is up; otherwise show logs and fail. if curl -sf "http://${IP}:8080/" >/dev/null 2>&1; then echo "Hello, Web app is up!" else echo "Website is not responding on port 8080" >&2 docker logs cg-app >&2 || true exit 1 fi ``` > **Note:** The Upload Files command copies the fixtures into the working directory (`cp $FIXTURES/*.* .`) so `Test.sh` sits alongside the student's `Dockerfile` and `app.py`. Depending on your CodeGrade version this variable may be `$UPLOADED_FILES` — if `Test.sh` is "not found", check that first. > **Tip:** To grade a specific endpoint instead of just "is it up", change what `Test.sh` prints the marker for, e.g. only `echo "Hello, Web app is up!"` when `curl .../add/3/5` returns `8`. > **Warning:** The container listens on port `8080`. If your students' apps use a different port, update both the `Dockerfile` (`EXPOSE`) and `Test.sh`. Reaching the container by its IP avoids host port-mapping conflicts when several submissions are graded in parallel. ### Test and publish your AutoTests Always test your configuration before releasing it to students. 1. Press **Build snapshot** at the bottom of the test block sidebar. 2. When prompted, upload the three reference files above as your **Test submission**. 3. Confirm the snapshot run fills the rubric category to 100% and that the output shows `PASS: website is up.` 4. Once you're happy, click **Publish snapshot** to make the tests available to students. > **Note:** Docker images and `pip install` can make the first build slow. If a snapshot times out, wrap the test **Script** in a **Timeout each** block and raise the limit (the default is 120 seconds). ### Conclusion You've built an assignment that grades a real, containerized web application end to end: CodeGrade installs Docker in Setup, builds the student's image, runs it, and verifies the site responds — all tied to a rubric. The same Setup + Script + Allow internet pattern extends to any Dockerized project. For more on the blocks used here, see the [AutoTest V2 Blocks](https://claude.ai/automatic-grading-guides/autotest-v2-blocks.md) guide, or reach out to our support team at . # R This tutorial will guide you through all the steps to create your first R assignment in CodeGrade. For this tutorial, we will consider the example below. The zip folder contains a completed solution, a template file, and some supporting files. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/fPz2ZqiaU4C5xdGzSV9j" %} **A quick note about this example**\ In this assignment, students have been asked to create and call a function in R that reads data about a record store, calculates the total revenue and average sales price per record, and prints this as output. ## Step 1: Create an assignment Firstly, we need an assignment to work with. Whether CodeGrade is integrated into your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step of setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.

Submission settings with the File uploader and Editor options enabled and a template file uploaded.

3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip Record\_Store.zip and upload the `analyze_record_store.r` file in the Template folder. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so your students don't have to create their own files. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [Submission Settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and clarify the grading requirements for your students. CodeGrade offers two types of rubric categories: * **Discrete category** - Specify points in discrete steps (e.g. 0, 5, or 10 points) * **Continuous category** - Specify points as a continuous scale (e.g. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 2 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
IO TestsDo you correctly calculate and print the total revenue and average sales price per record?0 - 80
Code Structure TestsDid you write your code in a function?0 - 20
## Step 4: Create automatic tests Automatic tests are really what allow your students to learn uniquely. The immediate feedback they get from the automatic tests allows them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial, we will set up IO and Code Structure tests. ### Step 4.1: Setup
Before creating the tests, we need to ensure that we have R installed correctly. 1. Navigate to the Setup tab under the AutoTest settings. 2. Add a Install R block to your configuration. 3. Add an Upload Files block to your configuration and upload `record_store_data.csv` 4. Navigate to the "Tests" tab of the AutoTest settings. 5. Add a script block to your configuration. Make the dataset available to the students' programs by adding the following command: ```bash mv $UPLOADED_FILES/record_store_data.csv ``` ### Step 4.3: Input/Output tests Input/Output tests, or "IO tests" for short, are a great way of checking a program's functionality by providing various input cases that should result in different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Add a **Connect rubric** block to your AutoTest configuration and select the "IO Tests" rubric. 2. Add an **IO test** block and nest it within the Connect rubric block. Run the student's programs with the command: ``` Rscript analyze_record_store.R ``` 3. Add two **Substring match** blocks to the IO Test block. The first will test the total revenue, and the second will test the average sales price per record. Place the following values in the Expected output field in their respective Substring match block: | Total revenue | Average sales price | | ------------- | ------------------- | | 43902.2 | 25.29 | {% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to you and your students what is being checked (e.g., "Does analyze\_record\_store.R produce the correct output?"). {% endhint %} ### Step 4.4: Code Structure Tests
Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax to reach their solution. AutoTest v2 offers the Code structure test block and corresponding Positive match and Negative match blocks for running Semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example, we want to check that the students have created a function in their program. Here is the Semgrep rule we will be using: ```yaml rules: - id: untitled_rule pattern: $FUNC <- function(...){...} message: Semgrep found a match languages: [R] severity: WARNING ``` 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "analyze\_record\_store.R". 4. Add a **Positive match** block to your test configuration and nest it within the Code structure test block. 5. Copy and paste the Semgrep rule in the code block above into the provided editor. Make sure to replace the template code provided. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to ensure your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. CodeGrade will prompt you to upload a Test submission the first time you build a snapshot. Use the "Click here to upload files" option or drag-and-drop `analyze_record_store.R` from the Solution folder in Record\_Store.zip and click "Submit". Once you're happy with your tests, you must publish them to your students so they can run on their submissions. Publish your tests by building a snapshot, and at the bottom of the pop-up modal, click the Publish snapshot button. ## Conclusion You have just built your first automatically graded assignment, and it is ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade offers. For more in-depth information about the product and the various workflows you can achieve, see [Learn more](/faq) or contact our support team at . # C ## Create your first autograded C assignment {% content-ref url="/pages/iSdvQe08LPiIovScq06T" %} [Create your first C assignment](/automatic-grading-guides/c/create-your-first-c-assignment) {% endcontent-ref %} {% content-ref url="/pages/zub7BCgLOW4cU3a1FHna" %} [Advanced C Autograding](/automatic-grading-guides/c/advanced-c-autograding) {% endcontent-ref %} # Create your first C assignment In this tutorial we will guide you through all the steps to create your very first C assignment in CodeGrade. For this tutorial we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/xmKYLi0UCK1rg1rvKLoH" %} **A quick note about this example**\ In this assignment, students have been asked to create a C program that prints the first 'n' numbers of the Fibonacci sequence starting from 1, where 'n' is determined by user input. ## Step 1: Create an assignment Firstly, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.

Submission settings with the File uploader and Editor options enabled and a template file uploaded.

3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Fibonacci.zip folder and upload Fibonacci/Template/Fibonacci.c as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [Submission Settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric category: * **Discrete category** - Specify points in discrete steps (eg. 0, 5 or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 3 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
Compilation testCheck that your program compiles without errors.0 - 20
IO testsCheck that your program produces the expected output given a specific input.0 - 40
Code structure testsCheck that your program uses the required code structures.0 - 40
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial we will be setting up three automatic tests: a compilation test, IO Tests, and Semgrep code structure tests. ### Step 4.1: Setup
Before actually creating the tests, we need to make sure that we have the correct version of our compiler, GCC, installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add the **Install GCC** block to your configuration. 3. Select the latest version of GCC using the drop-down menu. ### Step 4.2: Compilation test
Since C is a compiled language, we first have to compile our students' `.c` files to get an executable that we can run. We will also make the compilation step a graded component of the assignment. You can create compile steps like this using the **Script** block in AutoTest v2. This block is essentially a shell script that passes or fails depending on the exit code of the commands. 1. Navigate to the **Tests** tab under the AutoTest settings. 2. Add a **Connect Rubric** block to your configuration and select the "Compilation test" category from the drop-down menu. 3. Add a **Script** block to your configuration. In the editor provided type the following command to compile your students' programs.
```shell gcc -o Fibonacci.o Fibonacci.c ``` ### Step 4.3: Input/Output tests
Input/Output tests, or "IO tests" for short, are a great way of checking a program's functionality by providing various input cases that should result in different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Add a **Connect Rubric** block to your AutoTest configuration and select the "IO Tests" Rubric category from the drop-down menu. 2. Add an **IO test** block to your AutoTest configuration and nest it within the Connect Rubric block. Run the program with the command `./Fibonacci.o` in the provided editor. 3. Add five **Substring match** blocks to your AutoTest configuration and nest them inside of the IO test block. Make sure each match block ignores whitespaces. 4. For each Substring match block, copy one row from the table into the **Input** and **Expected output** fields in their respective match blocks: | Input | Expected output | | ----- | --------------- | | 1 | 1 | | 3 | 1 1 2 | | 5 | 1 1 2 3 5 | | 0 | invalid | | -1 | invalid | {% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to both you and your students what is actually being checked (e.g. "Does your program produce the correct output? - Input: 5"). {% endhint %} ### Step 4.4: Code structure tests

A graded Code structure test in AutoTest v2

Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax to reach their solution. AutoTest v2 offers the **Code structure test** block and its corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example, we want to check that the students have used a for loop and an if-statement in their program. Here are the Semgrep rules we will be using: {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: for($INT = $NUM; $COND; $UPDATE_INT){...} message: Semgrep found a match languages: [c] severity: WARNING ``` {% endcode %} {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: if($COND){...} message: Semgrep found a match languages: [c] severity: WARNING ``` {% endcode %} 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" category from the drop-down menu. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "Fibonacci.c". 4. Add two **Positive match** blocks to your test configuration and nest them within the Code structure test block 5. Copy and paste the semgrep rules in the code blocks above into the provided editor in each Positive match block. Make sure to delete the template provided. {% hint style="warning" %} The version of Semgrep available in the AutoTest v2 editor may not be the most recent. For this reason, rules that work in the Semgrep playground may not always work in the Code Structure test block. {% endhint %} ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the `Fibonacci/Solution/fibonacci.c` file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit" Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish to students** button. ## Conclusion You have just built your first automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](/faq) or reach out to our support team at . # Advanced C Autograding Discover the advanced autograding options for C assignments In this guide, we explore the advanced grading options available for C assignments. For more information about setting up a C assignment from scratch, see: {% content-ref url="/pages/iSdvQe08LPiIovScq06T" %} [Create your first C assignment](/automatic-grading-guides/c/create-your-first-c-assignment) {% endcontent-ref %} ## Check Unit Tests [Check](https://libcheck.github.io/check/) is an industry-standard unit testing framework for C. It is particularly useful for grading assignments that require students to write functions and classes. Unit testing with check offers several advantages over conventional IO testing including the ability to use assertions, parametrize test cases, and provide better feedback for students. Consider the example submission below: {% code title="bubble\_sort.h" %} ```c #ifndef BUBBLESORT_H #define BUBBLESORT_H void bubbleSort(int arr[], int n); #endif // BUBBLESORT_H ``` {% endcode %} {% code title="bubble\_sort.c" %} ```c #include "bubble_sort.h" void bubble_sort(int arr[], int n) { int i, j, temp; for (i = 0; i < n-1; i++) { for (j = 0; j < n-i-1; j++) { if (arr[j] > arr[j+1]) { // Swap arr[j] and arr[j+1] temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; } } } } ``` {% endcode %} In this example, students have been tasked with creating the `bubble_sort()` function. We can create robust test cases to check this submission using assertions from the check unit testing framework. {% code title="unit\_test.c" %} ```c #include #include "bubbleSort.h" START_TEST(test_bubbleSort_basic) { int arr[] = {5, 3, 8, 4, 2}; int expected[] = {2, 3, 4, 5, 8}; int n = 5; bubbleSort(arr, n); for (int i = 0; i < n; i++) { ck_assert_int_eq(arr[i], expected[i]); } } END_TEST START_TEST(test_bubbleSort_sorted) { int arr[] = {1, 2, 3, 4, 5}; int expected[] = {1, 2, 3, 4, 5}; int n = 5; bubbleSort(arr, n); for (int i = 0; i < n; i++) { ck_assert_int_eq(arr[i], expected[i]); } } END_TEST START_TEST(test_bubbleSort_reverse) { int arr[] = {9, 7, 5, 3, 1}; int expected[] = {1, 3, 5, 7, 9}; int n = 5; bubbleSort(arr, n); for (int i = 0; i < n; i++) { ck_assert_int_eq(arr[i], expected[i]); } } END_TEST START_TEST(test_bubbleSort_duplicates) { int arr[] = {4, 1, 3, 4, 2}; int expected[] = {1, 2, 3, 4, 4}; int n = 5; bubbleSort(arr, n); for (int i = 0; i < n; i++) { ck_assert_int_eq(arr[i], expected[i]); } } END_TEST Suite* bubbleSort_suite(void) { Suite *s; TCase *tc_core; s = suite_create("BubbleSort"); /* Core test case */ tc_core = tcase_create("Core"); tcase_add_test(tc_core, test_bubbleSort_basic); tcase_add_test(tc_core, test_bubbleSort_sorted); tcase_add_test(tc_core, test_bubbleSort_reverse); tcase_add_test(tc_core, test_bubbleSort_duplicates); suite_add_tcase(s, tc_core); return s; } int main(void) { int number_failed; Suite *s; SRunner *sr; s = bubbleSort_suite(); sr = srunner_create(s); srunner_set_xml(sr, "report.xml"); srunner_run_all(sr, CK_NORMAL); number_failed = srunner_ntests_failed(sr); srunner_free(sr); return (number_failed == 0) ? 0 : 1; } ``` {% endcode %} ### "cg junitxml" command We could simply compile and run the test cases to show students a simple pass or fail depending on the exit code of the tests. However, we wouldn't be able to award partial marks for each test case unless we parsed the results of the unit tests. To accomplish that, we can use the `cg junitxml` command. This command allows us to parse the number of test cases passed and the feedback from failed test cases from any unit test report written in the JUnit XML format. Running the command `cg junitxml --help` shows the following information: ``` This is a parser for generic unit test coverage reports in JUnit XML format. The parser can take in input multiple JUnit XML files. The final reported score is an aggregate of all the reports. Skipped test cases do not count towards the score. Example use: Input to the command (in a file called coverage.xml): ERROR: Expected: 6.0 but was: 5.0 Category: Checking returns - Multiplication File: /home/codegrade/student/calculator.py Line: 2 You would then call: cg junitxml coverage.xml Usage: cg junitxml XMLS... [flags] Flags: -h, --help help for junitxml --no-parse-weight Don't try to parse the weight from titles. --no-score Don't calculate and output the final score based on the results of the input data. -o, --output string The output file to to use. If this is a number it will use that file descriptor. (default "3") ``` The `cg junitxml` command works well in combination with the Custom Test block by displaying the results of the parsed JUnit XML report beautifully, making it easy to read and interpret, as shown in the image below.
Unfortunately, the check framework doesn't have a method for automatically outputting JUnit XML. Instead we can use the `srunner_set_xml()` command to generate an XML report (in check's own format) and use the following script to convert that into the JUnit XML format. {% code title="junit\_xml.py" %} ```c #!/usr/bin/env python3 # pylint: disable=missing-module-docstring,missing-function-docstring import os import sys import typing as t import subprocess import xml.etree.ElementTree as ET def _find_and_get_text(el: ET.Element, to_find: str) -> str: found = el.find(to_find) assert found is not None, ( 'Expected to find %s in %s, but could not find it' % (to_find, el.tag) ) return found.text or '' def parse_case(el: ET.Element) -> ET.Element: case = ET.Element('testcase') case.set('classname', _find_and_get_text(el, 'description')) case.set('name', _find_and_get_text(el, 'id')) # Sometimes the 'duration' attr is -1. case.set('time', str(max(0, float(_find_and_get_text(el, 'duration'))))) state = el.get('result') if state is not None and state != 'success': message = ET.Element(state) message.text = _find_and_get_text(el, 'message') case.append(message) return case def parse_suite(el: ET.Element) -> ET.Element: cases = [parse_case(c) for c in el.findall('test')] suite = ET.Element('testsuite') suite.set('name', _find_and_get_text(el, 'title')) suite.set('tests', str(len(cases))) suite.set( 'failures', str(sum(1 for c in cases if c.find('failure') is not None)) ) suite.set( 'errors', str(sum(1 for c in cases if c.find('error') is not None)) ) suite.set('time', str(sum(float(c.get('time') or '0') for c in cases))) suite.extend(cases) return suite def parse_tree(el: ET.Element) -> ET.ElementTree: assert el.tag == 'testsuites' suites = [parse_suite(s) for s in el.findall('suite')] tree = ET.Element('testsuites') tree.set('time', _find_and_get_text(el, 'duration')) tree.set('tests', str(sum(int(s.get('tests') or '0') for s in suites))) tree.set( 'failures', str(sum(int(s.get('failures') or '0') for s in suites)) ) tree.set('errors', str(sum(int(s.get('errors') or '0') for s in suites))) tree.extend(suites) return ET.ElementTree(tree) def run( test: t.List[str] ) -> t.NoReturn: """Run a Check suite and parse its results """ status = subprocess.run(test) # Remove namespaces in tag names. tree = ET.iterparse("report.xml") for _, el in tree: _, _, el.tag = el.tag.rpartition('}') root = tree.root # type: ignore parse_tree(root).write("report.xml") sys.exit(status) if __name__ == '__main__': run(sys.argv[1:]) ``` {% endcode %} ### Instructions
1. In the AutoTest settings, navigate to the **Setup** tab. 2. Add an **Install GCC** block to your setup configuration. 3. Add a Script block to your setup configuration and install check with the following command. ```bash sudo apt install check ``` 4. In the AutoTest settings, navigate to the **Tests** tab. 5. Add an Upload Files block. Upload `bubble_sort.h`, `unit_test.c`, and `junit_xml.py`. 6. Add a Script block. Move all the uploaded files to the student's directory and compile `bubble_sort.c` with the following commands. ```sh mv $UPLOADED_FILES/* . gcc -o unit_test unit_test.c bubbleSort.c -lcheck -lm -lpthread -lrt -lsubunit ``` 7. Add a Connect Rubric block and a Custom Test block. Nest the Custom Test block within the Connect Rubric block. Run the unit tests and parse the results with the following commands. ```bash python3 check.py ./unit_test cg junitxml junit.xml ``` 8. Build and publish your snapshot. ## Clang-tidy [Clang-tidy](https://clang.llvm.org/extra/clang-tidy/) is an industry-standard static analysis tool for C, C++, and Objective-C that allows you to diagnose and fix code style violations, interface misuse, and bugs. It is a useful tool for enforcing code styling best practices for beginner programmers. ### "cg comments" command Simply running clang-tidy would allow us to produce the default command line output. However, this wouldn't be particularly useful for students as they would have to spend time interpreting the output and would have to switch back and forth between the AutoTest output and their code. Instead, we can use the `cg comments` command to parse the output of clang-tidy and write the comments directly onto our students' code. The `cg comments` command will highlight each target line according to the severity of the comment and can be read by hovering over the line number with the mouse cursor. The comments are also placed on students' code in the editor, making it a powerful combination.
### Instructions
1. In the AutoTest settings, navigate to the **Setup** tab. 2. Add a Script block. Install clang-tidy using the following command. 
sudo apt update
   sudo apt install clang-tidy
3. In the AutoTest settings, navigate to the Tests tab. 4. Add a Connect Rubric block and a Custom Test block. Nest the Custom Test block in the Connect Rubric block. Run clang-tidy and parse the output using the following commands. ```bash clang-tidy bubble_sort.c --quiet --checks=* -- | cg comments \ '^(?P[^:]*):(?P[\d]+):(?P[\d]+): (?P\s*\S*): (?P.*)$' \ --origin clang-tidy \ --ignore-regex '^\s*$|^[^/].*$' ``` 5. Build and Publish your snapshot. # C\# ## Create your first autograded C# assignment {% content-ref url="/pages/mMtdUF4amK8wlACxi2nV" %} [Create your first C# assignment](/automatic-grading-guides/c-1/create-your-first-c-assignment) {% endcontent-ref %} ## Advanced grading options for C# assignments {% content-ref url="/pages/jB1rsQs0NgMonLE3RSIw" %} [Advanced C# autograding](/automatic-grading-guides/c-1/advanced-c-autograding) {% endcontent-ref %} # Create your first C# assignment In this tutorial we will guide you through all the steps to create your very first C# assignment in CodeGrade. For this tutorial we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/kBFKzWGP7OqEXAPC8VQe" %} **A quick note about this example**\ In this assignment, students have been asked to create a C# program that prints the first 'n' numbers of the Fibonacci sequence starting from 1, where 'n' is determined by user input. This program will be compiled and run using the .NET framework. ## Step 1: Create an assignment Firstly, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.

Submission settings with the File uploader and Editor options enabled and a template file uploaded.

3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Fibonacci.zip folder and upload the Template/Fibonacci.cs file as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [Submission Settings](/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric

Rubric categories

While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric category: * **Discrete category** - Specify points in discrete steps (eg. 0, 5 or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 3 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
Compilation testCheck that your program compiles without errors.0 - 20
IO testsCheck that your program produces the expected output given a specific input.0 - 40
Code structure testsCheck that your program uses the required code structures.0 - 40
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial we will be setting up three automatic tests: a Compilation test, IO tests, and Semgrep code structure tests. ### Step 4.1: Setup
Before actually creating the tests, we need to make sure that our testing environment is properly set up. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add the **Install .NET** block to your configuration and select the latest version of .NET using the drop-down menu. 3. Add an **Upload Files** block to your AutoTest configuration and upload the Fixtures/Fibonacci.csproj file ### Step 4.2: Compilation test
Since C# is a compiled language, we first have to compile our students' projects before we can run them. We will also make the compilation step a graded component of the assignment. You can create compile steps like this using the **Script** block in AutoTest v2. This block is essentially a shell script that passes or fails depending on the exit code of the commands. 1. Navigate to the **Tests** tab under the AutoTest settings. 2. Add a **Connect Rubric** block to your configuration and select the "Compilation test" category from the drop-down menu. 3. Add a **Script** block to your configuration. In the editor provided, type the following command to make Fibonacci.csproj available in the current directory and compile your students' programs.
mv $UPLOADED_FILES/Fibonacci.csproj .
   dotnet build --disable-build-servers
### Step 4.3: Input/Output tests
Input/Output tests, or "IO tests" for short, are a great way of checking a program's functionality by providing various input cases that should result in a different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. 1. Add a **Connect Rubric** block to your AutoTest configuration and select the "IO tests" rubric category. 2. Add an **IO test** block to your AutoTest configuration and nest it in the Connect Rubric block. 3. Run the the program with the command `dotnet run Fibonacci.cs` in the provided editor. 4. Add five **Substring match** blocks to your IO test block. For each one, make sure the "ignore whitespaces" option is selected. 5. Copy and paste the text below into the **Input** and **Expected output** fields: | Input | Expected output | | ----- | --------------- | | 1 | 1 | | 3 | 1 1 2 | | 5 | 1 1 2 3 5 | | 0 | invalid | | -1 | invalid | {% hint style="success" %} It's a good idea to give your tests descriptive names so that it's clear to both you and your students what is actually being checked (e.g. "Does your program produce the correct output? - Input: 5"). {% endhint %} ### Step 4.4: Code structure tests

A graded Code structure test in AutoTest v2

Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax for reaching their solution. AutoTest v2 offers the **Code structure test** block and it's corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example we want to check that the students have used a for loop and an if-statement in their program. Here are the Semgrep rules we will be using: {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: for (int $I = $I0; $COND2; $COND3) {...} message: Semgrep found a match languages: [csharp] severity: WARNING ``` {% endcode %} {% code lineNumbers="true" %} ```yaml rules: - id: untitled_rule pattern: if($COND){...} message: Semgrep found a match languages: [csharp] severity: WARNING ``` {% endcode %} 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" category from the drop-down menu. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "Fibonacci.cs". 4. Add two **Positive match** blocks to your test configuration and nest them within the Code structure test block 5. Copy and paste the semgrep rules in the code blocks above into the provided editor in each Positive match block. Make sure to delete the template provided. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the `Solution/Fibonacci.cpp` file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to **Upload Submission** option in the **General** settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit" Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish snapshot** button. ## Conclusion You have just built your first Automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](/faq) or reach out to our support team at . # Advanced C# autograding Discover the advanced autograding options available for C# assignments In this guide, we explore the advanced grading options available for C# assignments. For more information about setting up a C# assignment from scratch, see: {% content-ref url="/pages/mMtdUF4amK8wlACxi2nV" %} [Create your first C# assignment](/automatic-grading-guides/c-1/create-your-first-c-assignment) {% endcontent-ref %} ## Place Compile or Build errors as comments with Using the .NET framework, we compile the students' files using the \ command, which produces an output that potentially contains compilation errors or warnings. We can report and highlight these inline within the student submission using the custom CodeGrade command `cg comments` .
For example, to build a `dotnet console` project, we can use a Script Block that runs the following commands: ```bash cd ~/console_project # copy the student's files into the project directory cp $STUDENT/*.cs . # save the output produced when building the project OUTPUT=$(dotnet build --disable-build-servers /clp:NoSummary) # use the cg comments command to produce inline comments echo "$OUTPUT" | cg comments generic \ '^(?P[^\(]*)\((?P\d+),(?P\d+)\):\s*(?Perror|warning|info)\s+(?PCS\d+):(?P[^[]*).*$' \ --severities 'Error:error,Warning:warning' \ --origin "build" \ --ignore-parser-errors # exit the script according to the outcome of the build command echo "$OUTPUT" | sed '/warning/d' if [[ "$OUTPUT" == *"error"* ]]; then exit 1 fi ``` This will result in compilation errors or warnings being visible to the students within the student submission folder, as shown below:
## Unit Tests with xUnit xUnit is the standard unit testing tool within the .NET Framework and can be used to test .NET languages such as C#. xUnit is especially appropriate for grading assignments that require students to write functions and classes. xUnit unit tests offer several advantages over conventional IO tests including the ability to use assertions, parametrize test cases, and provide custom feedback for students. For an overview of the testing attributes available with xUnit, we recommend [this reference](https://xunit.net/docs/comparisons). \ Consider the following `Calculator.cs` submission that implements a simple arithmetic calculator: ```csharp using System; public class Calculator { public double Add(double a, double b) { return a + b; } public double Subtract(double a, double b) { return a - b; } public double Multiply(double a, double b) { return a * b; } public double Divide(double a, double b) { if (b == 0) { Console.WriteLine("Error: Division by zero is not allowed."); return double.NaN; } return a / b; } } ``` Our tests are defined in the following `CalculatorTests.cs` file: ```csharp using Xunit; using System; namespace CalculatorTests { public class CalculatorTests { private Calculator calculator = new Calculator(); [Fact(DisplayName = "[5] Addition Test")] public void AdditionTest() { double result = calculator.Add(5, 3); Assert.Equal(8, result); } [Fact(DisplayName = "[3] Subtraction Test")] public void SubtractionTest() { double result = calculator.Subtract(5, 3); Assert.Equal(1, result); } [Fact(DisplayName = "[5] Multiplication Test")] public void MultiplicationTest() { double result = calculator.Multiply(5, 3); Assert.Equal(15, result); } [Fact(DisplayName = "[3] Division Test")] public void DivisionTest() { double result = calculator.Divide(6, 3); Assert.Equal(2, result); } [Fact(DisplayName = "Division by Zero Test")] public void DivisionByZeroTest() { Assert.Equal(double.NaN, calculator.Divide(6, 0)); } } } ``` As displayed above, we can use the xUnit's `Fact` attribute to set a custom name for each test. Moreover, by using naming the test as `"[n] My Custom Test Name"`, we can also set a custom weight equal to `n` , with the default weight being 1. ### Setup In order to run unit tests with xUnit, in the Setup Phase we first have to install dotnet using the corresponding block and create an xUnit project.
After installing dotnet, use a Script Block and type the following bash script: {% code fullWidth="true" %} ```bash # create a folder with an xUnit project mkdir xunit_project cd xunit_project dotnet new xunit # remove the template testing class created by default rm UnitTest1.cs # install tool to create a test xml report in the junit format dotnet add package XunitXml.TestLogger --version 3.1.20 dotnet tool install --global dotnet-xunit-to-junit --version 6.0.0 # make the installed tool available from the command line by adding it to the path echo 'export PATH=$PATH:~/.dotnet/tools' >> ~/.cg_bash_env ``` {% endcode %} {% hint style="danger" %} If the class you want to test contains a `main` method you should add the following line at the bottom of the script: {% endhint %} ```bash # Modify the xUnit project configuration so that dotnet does not create an additional Main Method for the Unit Test class sed -i '//a false' xunit_project.csproj ``` ### Tests #### Project Build Once we are in the Tests phase, before running the actual tests, we have to: * Upload the fixture file that defines the xUnit tests; * Move both the xUnit testing file and the student submission files to the xUnit project folder; * Compile the student's code by building the xUnit Project.
After using an Upload File block to upload the xUnit testing file, use a Script Block to run the following script: {% code fullWidth="true" %} ```bash # move to the xunit project folder cd ~/xunit_project # add both the submission and testing files to the xUnit project cp $STUDENT/*.cs . cp $FIXTURES/*.cs . # build the project OUTPUT=$(dotnet build --disable-build-servers /clp:NoSummary) # display compilation comments inline echo "$OUTPUT" | sed 's|/Files|/Student|g' | cg comments generic \ '^(?P[^\(]*)\((?P\d+),(?P\d+)\):\s*(?Perror|warning|info)\s+(?PCS\d+):(?P[^[]*).*$' \ --severities 'Error:error,Warning:warning' \ --origin "build" \ --ignore-parser-errors # exit the script according to the outcome of the build command echo "$OUTPUT" | sed '/warning/d' if [[ "$OUTPUT" == *"error"* ]]; then exit 1 fi ``` {% endcode %} Notice that: * you can hide the script's content from the student using a Hide block as shown in the image above. * we can report potential compilation errors inline into the student's submission, as explained at the [top of this page](#place-compile-or-build-errors-as-comments-with). #### Test execution We are finally ready to run our tests. For this, we can use a Custom Test Block.
The Custom Test Block runs the following commands: {% code fullWidth="true" %} ```bash # move to the xUnit project folder cd ~/xunit_project # run the xUnit Tests and log the test results in junit xml format dotnet test --disable-build-servers --logger:"xunit;LogFilePath=results.xml" # convert the xml in the junit format dotnet xunit-to-junit "results.xml" "junitResults.xml" # use the cg command to parse the test result and display the result to the student cg junitxml junitResults.xml # shut down the dotnet background process dotnet build-server shutdown ``` {% endcode %} As shown in the image above, you can optionally: * hide the configuration of the Custom Test Block from the student; * run the tests only if the Compilation step succeeded using a Run-If Block. The results of the tests will be shown to the student as below:
# C++ ### Create your first autograded C++ assignment {% content-ref url="/pages/KbSE5Rx8a1S22kb9Nepu" %} [Create your first C++ assignment](/automatic-grading-guides/c++/create-your-first-c++-assignment) {% endcontent-ref %} ### Advanced grading options for C++ assignments {% content-ref url="/pages/wL5tWUnbM6HVaCymN0IM" %} [Advanced C++ autograding](/automatic-grading-guides/c++/advanced-c++-autograding) {% endcontent-ref %} # Create your first C++ assignment In this tutorial, we will guide you through all the steps to create your very first C++ assignment in CodeGrade. For this tutorial, we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/Ia6GiSSQxFRHOxb0yuyL" %} **A quick note about this example**\ In this assignment, students have been asked to create a C++ program that prints the first 'n' numbers of the Fibonacci sequence starting from 1, where 'n' is determined by user input. ## Step 1: Create an assignment First, we need an assignment to work with. Whether CodeGrade is integrated with your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.
3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Calculator.zip folder and upload `Fibonacci/Template/Fibonacci.cpp` as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [submission settings](https://help.codegrade.com/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric
While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and to clarify the grading requirements to your students. CodeGrade offers two types of rubric categories: * **Discrete category** - Specify points in discrete steps (eg. 0, 5, or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create New Rubric**. 3. Create 3 new **Continuous Categories**. Set the parameters as follows:
NameDescriptionPoints
Compile testCheck that your code compiles without errors.0 - 20
Input/Output testsCheck that your program produces the correct output depending on the provided input.0 - 40
Code structure testsCheck that you have used the expected code structures0 - 40
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allows them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial, we will be setting up two types of automatic tests: IO tests and Semgrep code structure tests. ### Step 4.1: Setup
Before actually creating the tests, we need to make sure we have the correct version of our compiler, GCC, installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add an **Install GCC** block to your configuration. 3. Select the latest version of GCC using the drop-down menu. ### Step 4.2: Compile test
As with any compiled language like C++, each student's program must be compiled before we can run any other tests on it. We will also grade this test using the "Compile test" rubric category : 1. Navigate to the **Tests** tab under the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration and select the "Compile test" rubric category from the drop-down menu. 3. Add a **Script** block to your AutoTest configuration and nest it within the Connect rubric block. 4. In the editor provided, add the following bash commands: ``` g++ -o Fibonacci Fibonacci.cpp ``` ### Step 4.3: Input/Output tests Input/Output tests (or "IO tests" for short) are a great way of checking a program's functionality by providing various input cases that should result in a different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. For more information about these blocks, see [Create an IO Test](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/create-an-io-test).
1. Navigate to the **Tests** tab of the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration. Select the "IO tests" rubric category from the drop-down menu. 3. Add an **IO test** block to your AutoTest configuration and nest it inside the Connect Rubric block. 4. In the editor provided use the command `./Fibonacci` to run the program. 5. Add five **Substring match** blocks to your AutoTest configuration and nest them inside the IO test block. Make sure each match block is case insensitive and ignores whitespace. 6. Copy and paste the info from the table below into the **Input** and **Expected output** fields in their respective match block: | Input | Expected output | | ----- | ---------------------------- | | 1 | 1 | | 3 |

1
1
2

| | 5 |

1
1
2
3
5

| | 0 | invalid | | -1 | invalid | ### Step 4.5: Code structure tests
Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax to reach their solution. AutoTest v2 offers the **Code structure test** block and its corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more information about these test blocks, see [Create a Code Structure Test](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/code-structure-test). For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example, we want to check that the students have the add function in their program. Here are the Semgrep rules we will be using: ```yaml rules: - id: untitled_rule pattern: for(...){...} message: Semgrep found a match languages: [cpp] severity: WARNING ``` ```yaml rules: - id: untitled_rule pattern: if(...){...} message: Semgrep found a match languages: [cpp] severity: WARNING ``` 1. Add a **Connect rubric block** to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code structure test** block to your test configuration and nest it within the Connect rubric block. 3. In the **Student file** input field, write "Fibonacci.cpp". 4. Add two **Positive match** blocks to your test configuration and nest them within the Code structure test block 5. Copy and paste the semgrep rules in the code blocks above into the provided editor in each Positive match block. Make sure to delete the template provided. {% hint style="warning" %} The most recent version of Semgrep may not be available in AutoTest v2. For this reason, rules that work in the Semgrep playground may not always work in the Code Structure test block. {% endhint %} ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the `Fibonacci/Solution/Fibonacci.cpp` file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit". Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish to students** button. Once it is published, AutoTest V2 is available for students through the editor and on the submission page. ## Conclusion You have just built your first automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](https://help.codegrade.com/faq) or reach out to our support team at . # Advanced C++ autograding Discover the advanced autograding options available for C++ assignments In this guide, we explore the advanced grading options available for C++ assignments. For more information about setting up a C++ assignment from scratch, see {% content-ref url="/pages/KbSE5Rx8a1S22kb9Nepu" %} [Create your first C++ assignment](/automatic-grading-guides/c++/create-your-first-c++-assignment) {% endcontent-ref %} ## Google Test Google Test is an industry-standard unit testing framework for C++. It is particularly useful for grading assignments that require students to write functions and classes. Using C++ unit tests offers several advantages over conventional IO tests, including the ability to use assertions, parametrize test cases, and provide better feedback for students. Consider the example submission below, which consists of a simple C++ class and its associated header file: {% tabs %} {% tab title="Calculator.cpp" %} {% code title="Calculator.cpp" %} ```cpp #include #include double add(double a, double b) { return a + b; } double subtract(double a, double b) { return a - b; } double multiply(double a, double b) { return a * b; } double divide(double a, double b) { if (b == 0) throw std::runtime_error("Division by zero error"); return a / b; } ``` {% endcode %} {% endtab %} {% tab title="Calculator.hpp" %} {% code title="Calculator.hpp" %} ```cpp #include #include double add(double, double); double subtract(double, double); double multiply(double, double ); double divide(double, double b); ``` {% endcode %} {% endtab %} {% endtabs %} In the following test file, we can define and create robust tests using Google Test assertions:
TestCalculator.cpp {% code title="TestCalculator.cpp" %} ```cpp #include #include #include "Calculator.hpp" // Test for Addition TEST(CalculatorTests, AddTest) { RecordProperty("cg_name", "Test Addition of Two Numbers"); EXPECT_DOUBLE_EQ(add(5, 3), 8) << "Addition should correctly add two numbers"; } // Test for Subtraction TEST(CalculatorTests, SubtractTest) { RecordProperty("cg_name", "Test Subtraction of Two Numbers"); EXPECT_DOUBLE_EQ(subtract(5, 3), 2) << "Subtraction should correctly subtract second number from first"; } // Test for Multiplication TEST(CalculatorTests, MultiplyTest) { RecordProperty("cg_name", "Test Multiplication of Two Numbers"); EXPECT_DOUBLE_EQ(multiply(5, 3), 15) << "Multiplication should correctly multiply two numbers"; } // Test for Division TEST(CalculatorTests, DivideTest) { RecordProperty("cg_name", "Test Division of Two Numbers"); EXPECT_DOUBLE_EQ(divide(10, 2), 5) << "Division should correctly divide first number by second"; } // Test for Division by Zero TEST(CalculatorTests, DivideByZeroTest) { RecordProperty("cg_name", "Test Division by Zero Throws Error"); EXPECT_THROW(divide(10, 0), std::runtime_error) << "Division by zero should throw a runtime_error"; } int main(int argc, char **argv) { testing::InitGoogleTest(&argc, argv); return RUN_ALL_TESTS(); } ``` {% endcode %} Notice the use of: * CodeGrade's builtin annotator `RecordProperty` that enables us to give custom names to our tests; * custom error messages that are displayed in case of failure of any `EXPECT` assertion.
### AutoTest Setup As explained in [our previous guide](https://help.codegrade.com/automatic-grading-guides/c++/pages/KbSE5Rx8a1S22kb9Nepu#step-4.1-setup), we first need to install a C++ compiler. Our second step is to use a Script Block to install Google Test through the following Bash script: ```bash sudo apt install cmake git clone https://github.com/google/googletest.git -b v1.14.0 cd googletest # Main directory of the cloned repository. mkdir build # Create a directory to hold the build output. cd build cmake .. # Generate native build scripts for GoogleTest. make sudo make install # Install in /usr/local/ by default ```
### AutoTest Tests In the Tests phase, we first need to use an Upload Files block to upload the following files: * the testing file `TestCalculator.cpp` defined above; * the CMake file `CMakeLists.txt` used to compile our tests: 
cmake_minimum_required(VERSION 2.6)
project(Test)
IF(POLICY CMP0012)
    CMAKE_POLICY(SET CMP0012 NEW)
ENDIF()
# Locate GTest
find_package(GTest REQUIRED)
find_package(Threads REQUIRED)
include_directories(${GTEST_INCLUDE_DIRS})

# Link runTests with what we want to test and the GTest and pthread library
add_executable(runTests TestCalculator Calculator)
target_link_libraries(runTests ${GTEST_LIBRARIES} pthread)
{% hint style="warning" %} In line 12, beside `runTests` we list the `cpp` files that need to be compiled together in order to run our tests, namely the testing file `TestCalculator.cpp` and our submission file `Calculator.cpp` . {% endhint %} {% hint style="info" %} **When the student's file includes a `main()` function** If the submitted file defines its own `main()`, you'll get a duplicate symbol linking error. To fix this, you can wrap the student's `main` at link time using the GNU linker." Add to `TestCalculator.cpp`: ```cpp extern "C" int __wrap_main(int x, char *y[]) { testing::InitGoogleTest(&x, y); return RUN_ALL_TESTS(); } ``` Add to `CMakeLists.txt`: ```cmake # Wrap main() from student code to avoid linking conflicts with GoogleTest set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--wrap=main") ``` {% endhint %} #### Tests Compilation We are now ready to compile our tests using a Script Block with the following script: ```bash set -e cp -r $UPLOADED_FILES/* . cmake -D CMAKE_CXX_COMPILER=clang++ CMakeLists.txt>/dev/null make >/dev/null ``` #### Tests Execution To finally run our tests we can use a Custom Test Block with the following script: ```bash ./runTests --gtest_output=json>/dev/null res=$? if [[ $res == 0 ]] || [[ $res == 1 ]]; then cg gtest parse test_detail.json else echo "Your tests did not execute correctly." exit $res fi ```

An AutoTest configuration to run Google Test.

{% hint style="info" %} You can use a Connect Rubric Block to connect the tests to a rubric category. {% endhint %} {% hint style="info" %} You can hide the configuration of the blocks from the student. {% endhint %} {% hint style="info" %} To enhance the AutoTest clarity for students, it can be useful to run the Tests conditionally on the success of the compilation block. {% endhint %} ## Clang-Tidy Clang-Tidy is an industry-standard linter for C++ that allows you to either enforce rules from various standard coding styles or to create your own set of rules. It is a useful tool for enforcing code styling best practices for beginner programmers. ### AutoTest Setup In the AutoTest Setup phase, in addition to typically installing a C++ compiler, we need to install Clang-Tidy with a simple Script Block that runs the following command: ```bash sudo apt-get update && sudo apt-get install clang-tidy ```
### AutoTest Tests Running Clang-Tidy is done through the use of a Custom Test Block that runs the following script: ```bash clang-tidy Calculator.cpp -checks=* 2>/dev/null | cg comments \ '^(?P/[^:]*):(?P\d+):(?P\d+): (?P[^:]*): (?P.*) (\[(?P[^\]]*)\])?$' \ --origin clang-tidy \ --ignore-regex '^([^/])' \ --severities 'note:info,remark:info' \ --deduction-warning '2' \ --ignore-parser-errors ``` This script first runs Clang-Tidy enforcing all the available checks through the option `-checks=*`, and then processes its output through the cg comments command. This will make such that all the coding style violations will be reported inline within the student's files in the submission folder.
{% hint style="info" %} You can select which checks to include/exclude among all the available ones. For this, we refer to the [Clang Tidy documentation.](https://clang.llvm.org/extra/clang-tidy/) {% endhint %} {% hint style="info" %} You can use a Connect Rubric Block to connect the tests to a rubric category. For a graded test you can decide the penalties to apply for each kind of violation (error, warning, etc...); for more information read here about the `cg comments` command. {% endhint %} {% hint style="info" %} You can hide the configuration of the blocks from the student. {% endhint %}

Coding Style violations are reported inline within the student's submission file.

## Conclusion These advanced C++ testing options open the door to more rigorous testing methods and better student feedback. However, these are just the most commonly used methods. There are many more testing tools available that you can use with some simple setup. For more information, contact us at . # PHP ### Create your first autograded PHP assignment {% content-ref url="/pages/fo3z1viXyMh3THTb2fOT" %} [Create your first PHP assignment](/automatic-grading-guides/php/create-your-first-php-assignment) {% endcontent-ref %} ### Advanced grading options for PHP assignments {% content-ref url="/pages/RB82etAo0PjZvcgQVeoM" %} [Advanced PHP autograding](/automatic-grading-guides/php/advanced-php-autograding) {% endcontent-ref %} # Create your first PHP assignment Create your first automatically graded PHP assignment using CodeGrade's AutoTest v2 In this tutorial we will guide you through all the steps to create your very first PHP assignment in CodeGrade. For this tutorial we will be considering the example below. The zip folder contains a completed solution and a template file. Please make sure to save this file somewhere accessible on your computer: {% file src="/files/5DVli5wxlgn5BeFoOI5K" %} **A quick note about this example** In this assignment, students have been asked to create a PHP program that prints the first 'n' numbers of the Fibonacci sequence, where 'n' is determined by user input. ## Step 1: Create an assignment First, we need an assignment to work with. Whether CodeGrade is integrated in your LMS or you are using CodeGrade Standalone, you can follow the steps in the guide below to get started: {% content-ref url="/pages/-MLxNHbt2dT3s4i1eyTZ" %} [Create assignment](/setup-assignment/build-assignment/creating-an-assignment) {% endcontent-ref %} ## Step 2: Configure submission settings The first step in setting up an assignment is to ensure that students can submit their code correctly. Because this example is straightforward and only requires a single solution file, we will offer the students the option to code in the CodeGrade Editor alongside the default File Uploader option. 1. Navigate to the **Assignment Settings** by using the **"**:gear: Assignment settings" link at the right-hand side of the page menu on the submissions page.
2. Find the **Submission settings** in the corresponding container under the **General** tab.
3. Enable the **File uploader** and **Editor** submission methods by clicking the respective check boxes. 4. Enabling the Editor will reveal the **Template files** option. Unzip the Fibonacci.zip folder and upload Fibonacci/Template/fibonacci.php as the template. 5. Finalize your settings by clicking the "Submit" button. {% hint style="success" %} It is good practice to provide a template file when enabling the Editor so that your students don't have to create their own file. This also avoids the risk of students submitting a file with the wrong name. {% endhint %} For more information about submission methods, see [submission settings](https://help.codegrade.com/setup-assignment/build-assignment/general-settings#step-2-set-submission-settings). ## Step 3: Create a rubric
While you can always grade assignments directly by manually setting the **Final grade**, It's only possible to award points for your automatic tests with **Rubrics**. Rubrics also allow you to standardize the grading scheme for graders and they allow you to clarify the grading requirements to your students. CodeGrade offers two types of rubric category: * **Discrete category** - Specify points in discrete steps (eg. 0, 5 or 10 points) * **Continuous category** - Specify points as a continuous scale (eg. 0 - 10) 1. Navigate to the **Rubric** tab on the **Assignment Management** page. 2. Select **Create new rubric**. 3. Create 2 new **Continuous Categories**. Set the parameters as follows:
Category nameDescriptionMin - Max points
IO testsCheck that your program produces the expected output given a specific input.0 - 75
Code structure testsCheck that your program uses the required code structures.0 - 25
## Step 4: Create automatic tests Automatic tests are really what gives your students the opportunity to learn in a unique way. The immediate feedback they get from the automatic tests allow them to learn through trial and error by applying the instant feedback and resubmitting their work. For this tutorial we will be setting up two automatic tests: IO tests and Semgrep code structure tests. ### Step 4.1: Setup
Before actually creating the tests, we need to make sure we have the correct version of PHP installed. 1. Navigate to the **Setup** tab under the AutoTest settings. 2. Add a **Script** block to your configuration, and input the following commands to install php: ```bash sudo apt update && sudo apt upgrade sudo apt install software-properties-common ca-certificates lsb-release apt-transport-https LC_ALL=C.UTF-8 sudo add-apt-repository ppa:ondrej/php sudo apt update sudo apt install php8.2 ``` ### Step 4.2: Input/Output tests
Input/Output tests (or "IO tests" for short) are a great way of checking a program's functionality by providing various input cases that should result in a different outputs. You can create IO test cases in AutoTest v2 using the **IO Test** wrapper block and its corresponding **Full match**, **Substring match**, and **Regex match** blocks. For more information about these blocks, see [Create an IO Test](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/create-an-io-test). 1. Navigate to the **Tests** tab of the AutoTest settings. 2. Add a **Connect rubric** block to your AutoTest configuration. Select the "IO tests" rubric category from the drop-down menu. 3. Add an **IO test** block to your AutoTest configuration and nest it inside the Connect Rubric block. 4. In the editor provided use the command `php fibonacci.php` to run the program. 5. Add five **Substring match** blocks to your AutoTest configuration and nest them inside of the IO test block. Make sure each match block is case insensitive and ignores whitespace. 6. Copy and paste the info from the table below into the **Input** and **Expected output** fields in their respective match block: | Input | Expected output | | ----- | ---------------------------- | | 1 | 1 | | 3 |

1
1
2

| | 5 |

1
1
2
3
5

| | 0 | invalid | | -1 | invalid | ### Step 4.3: Code structure tests
Semgrep is a code structure testing framework that is excellent for checking if students have used a specific syntax for reaching their solution. AutoTest v2 offers the **Code structure test** block and it's corresponding **Positive match** and **Negative match** blocks for running semgrep tests. For more information about these test blocks, see [Create a Code Structure Test](https://help.codegrade.com/faq/automatic-grading-questions/autotest-v2/code-structure-test). For more details on setting up Code Structure Tests, read [our dedicated guide](/automatic-grading-guides/code-structure-tests-with-semgrep). For this example we want to check that the students have used a for loop in their program. Here is the Semgrep rule we will be using: ```yaml rules: - id: untitled_rule pattern: | for($A=$B;$COND;$ACT){ ... } message: Semgrep found a match languages: [php] severity: WARNING ``` 1. Add a **Connect Rubric** block to your test configuration and select the "Code structure tests" rubric category. 2. Add a **Code Structure Test** block to your test configuration and nest it within the Connect Rubric block. 3. In the **Student file** input field, write "fibonacci.php". 4. Add a **Positive Match** block to your test configuration and nest it within the Code structure test block 5. Copy and paste the semgrep rule above into the Positive Match block. Make sure to delete the template provided. ## Step 5: Test and publish your AutoTests It's important to test your AutoTest configuration before running it on students' submissions to make sure that your tests are running as expected and to check for edge cases. This is easy to do with **Snapshots**. When you're ready to check your tests, simply press the **Build snapshot** button at the bottom of the test block sidebar. This will make a test run of your AutoTest configuration on your Test submission. The first time you build a snapshot, CodeGrade will prompt you to upload a **Test submission**. Use the "Click here to upload files" option or drag-and-drop the Fibonacci/Solution/fibonacci.php file that you downloaded at the beginning of this tutorial and click "Submit". If in the future you need to re-upload your test submission, you can do so by going to Upload Submission option in the General Settings tab. Just upload your file and check the "Test Submission" option before clicking "Submit". Once you're happy with your tests, you need to publish them to your students for them to run on their submissions. Publish your tests by building a snapshot and at the bottom of the pop-up modal click the **Publish to students** button. Once it is published, AutoTest V2 is available for students through the editor and on the submission page. ## Conclusion You have just built your first automatically graded assignment, ready for your students to begin submitting work! This guide is designed to get you started with a completed assignment but doesn't go into explicit detail about all of the features CodeGrade has to offer. For more in-depth information about the product and the various workflows that you can achieve, see [Learn more](https://help.codegrade.com/faq) or reach out to our support team at . # Advanced PHP autograding Discover the advanced autograding options available for PHP assignments In this guide, we explore the advanced grading options available for PHP assignments. For more information about setting up a PHP assignment from scratch, see: {% content-ref url="/pages/fo3z1viXyMh3THTb2fOT" %} [Create your first PHP assignment](/automatic-grading-guides/php/create-your-first-php-assignment) {% endcontent-ref %} ## Unit tests with PHPUnit PHPUnit is an industry-standard unit testing framework for PHP. It is particularly useful for grading assignments that require students to write functions and classes. Using PHPUnit unit tests offers several advantages over conventional IO tests, including the ability to use assertions, parametrize test cases, and provide better feedback for students. For further documentation about PHPUnit we recommend [this online resource](https://phpunit.de/documentation.html). \ Consider the example submission `Fibonacci.php` file below: ```php assertEquals(0, ComputeFibonacci(0)); } public function testFibonacciOne() { $this->assertEquals(1, ComputeFibonacci(5)); } public function testFibonacciTwo() { $this->assertEquals(1, ComputeFibonacci(2)); } public function testFibonacciThree() { $this->assertEquals(2, ComputeFibonacci(3)); } public function testFibonacciTen() { $this->assertEquals(55, ComputeFibonacci(10)); } public function testFibonacciTwenty() { $this->assertEquals(6765, ComputeFibonacci(20)); } } ``` ### Setup: Install PHPUnit After installing PHP as described in the previous guide, we can use Script Block to install Composer, a PHP dependency management framework, and then PHPUnit.
In the `Install php dependecies` Script block above, we run the following Bash script: {% code fullWidth="true" %} ```shell set -e # install composer yes | sudo apt install php-cli unzip curl -sS https://getcomposer.org/installer -o /tmp/composer-setup.php HASH=`curl -sS https://composer.github.io/installer.sig` php -r "if (hash_file('SHA384', '/tmp/composer-setup.php') === '$HASH') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" sudo php /tmp/composer-setup.php --install-dir=/usr/local/bin --filename=composer # install php dependencies sudo apt-get update sudo apt-get install php-xml sudo apt-get install php-mbstring # install php tools composer require --dev phpunit/phpunit:^9.5 # add the tools to the path echo 'export PATH=$PATH:~/vendor/phpunit' >> ~/.cg_bash_env echo 'export PATH=$PATH:~/vendor/bin' >> ~/.cg_bash_env ``` {% endcode %} ### Test execution Once we are done with the Setup Phase in AutoTest, we can move on to the next Tests phase. Here, we first need to upload our testing file `FibonacciTests.php` using an Upload Files Block. We can then run our unit tests using a Custom Test Block as shown below:
The Custom Test Block runs the following commands: {% code fullWidth="true" %} ```bash set -e # move the necessary files to the current directory cp $UPLOADED_FILES/* . mv ~/composer.json . # wrap a bash script around phpunit (needed to handle the exit code returned by phpunit) cat < script.sh phpunit --log-junit report.xml FibonacciTests.php cg junitxml report.xml exit 0 EOF # run the script that calls phpunit chmod +x script.sh ./script.sh ``` {% endcode %} Notice that you can hide these commands from the students using a Hide Block. The results of the unit tests will be shown to the student as below:
## Code Quality Tests with PHP\_CodeSniffer PHP\_CodeSniffer is an industry-standard linter for PHP that allows you to enforce rules from various coding standards. It is a useful tool for enforcing code styling best practices for beginner programmers. To read more about PHP\_CodeSniffer, we recommend [this online resource](https://github.com/squizlabs/PHP_CodeSniffer/wiki/Configuration-Options#setting-the-default-coding-standard). ### Setup: Install php\_codesniffer We can repeat the same steps described in the section above for installing PHPUnit, just now the installing script is slightly different: ```bash set -e # install composer yes | sudo apt install php-cli unzip curl -sS https://getcomposer.org/installer -o /tmp/composer-setup.php HASH=`curl -sS https://composer.github.io/installer.sig` php -r "if (hash_file('SHA384', '/tmp/composer-setup.php') === '$HASH') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" sudo php /tmp/composer-setup.php --install-dir=/usr/local/bin --filename=composer # install php dependencies sudo apt-get update sudo apt-get install php-xml sudo apt-get install php-mbstring # install php tools composer require --dev squizlabs/php_codesniffer # add the tools to the path echo 'export PATH=$PATH:~/vendor/phpunit' >> ~/.cg_bash_env echo 'export PATH=$PATH:~/vendor/bin' >> ~/.cg_bash_env ``` ### Test execution We can run PHP\_Codesniffer on a student file, `Fibonacci`.php\` in the example below, through a Custom Test Block:
Within the Custom Test Block, we run the following script: {% code fullWidth="true" %} ```bash set -e # wrap a bash script around phpcs (needed to handle the exit code returned by phpcs) cat < script.sh phpcs --report=checkstyle --standard=PSR12 ./Fibonacci.php > report.xml cg checkstyle parse report.xml exit 0 EOF # run the script that calls phpcs chmod +x script.sh ./script.sh ``` {% endcode %} Running the Code Quality Test will produce inline comments within the student submission, as shown below:
# Additional Languages Is your programming language not listed in our Automatic Grading Guides?\ No problem — CodeGrade is designed to support flexible programming environments. {% hint style="info" %} CodeGrade environments run on *Ubuntu*, and instructors can choose between: * *Ubuntu 20.04* * *Ubuntu 24.04* {% endhint %} Any programming language that runs on Ubuntu can typically be used in CodeGrade. This means that even if your language is not listed in our documentation, it may still work with a custom configuration. ### Why aren't all languages listed? Many automated grading tools only support a fixed list of languages. CodeGrade takes a different approach by allowing customizable environments, making it possible to support a much wider range of programming languages and course requirements. ### Request a Language Setup If you would like to use a language that is not currently listed in our guides, our support team can help configure the environment for your assignment. {% hint style="success" %} Reach out to **** and let us know which language or runtime you would like to use. Our team will help you get everything set up. {% endhint %} # Code Structure Tests with Semgrep Learn how to create effective tests to check that your students employ the required coding patterns. [Semgrep ](https://semgrep.dev/)is a static code analysis tool that can check if students have used a specific syntax or coding pattern to reach their solution. AutoTest offers the **Code structure test** block and its corresponding **Positive match** and **Negative match** blocks for running Semgrep tests. These blocks are configured by specifying Semgrep patterns. To get familiar with Semgrep patterns, we highly recommend the following resources: * Semgrep's [online documentation](https://semgrep.dev/learn/basics/) has many interactive tutorials and examples. * Semgrep's [online playground](https://semgrep.dev/playground/new) is helpful for quickly developing and debugging your patterns. This page is divided into two parts. The first part walks you through creating a Code Structure Test in AutoTest. The second part is a collection of Semgrep patterns that you can use out of the box for your tests. ## Setting up a Code Structure Test in AutoTest Follow these steps in the AutoTest Editor to create a Code Structure Test: * Add a **Code Structure Test** Block to your AutoTest configuration. * In the **Student file** input field, write the name of the file you want to check. The default value will check every file in the Student Directory. * Add an inner **Positive** or **Negative Match Block** within the **Code Structure Test** Block. The Positive Match will pass if the expected coding pattern is found in the target file, while for the Negative Match, a test passes if the expected coding pattern is not found. * Edit the template configuration of the Match Block you want to configure.
More about the Match Block configuration The template configuration looks like the following: ```yaml # Use the semgrep playground to create a rule, then copy the result you find # in the `advanced` tab into this block. https://semgrep.dev/editor rules: - id: untitled_rule pattern: YOUR_PATTERN message: Semgrep found a match languages: [THE_PROGRAMMING_LANGUAGE] severity: WARNING ``` Note the link to the Semgrep editor in the comment. We recommend developing your Semgrep pattern in this environment and then copy-pasting it back to CodeGrade. You only need to edit the **pattern** and **languages** field. * for the pattern, copy and paste your Semgrep rule. * for the languages, type one of the supported languages within square brackets. Below is an example of a working configuration to check the presence of a function named **myFunction** in Python: ```yaml # Use the semgrep playground to create a rule, then copy the result you find # in the `advanced` tab into this block. https://semgrep.dev/editor rules: - id: untitled_rule pattern: | def myFunction(...): ... message: Semgrep found a match languages: [python] severity: WARNING ```
{% hint style="info" %} Wrap a **Hide Configuration Block** around your **Code Structure Test Block** to prevent students from seeing the configuration of your **Match Block**. This can avoid confusion. {% endhint %}

A Code Structure Test in AutoTest. This test is connected to a Rubric Category and its configuration is not visible to the students.

## Tips and tricks * **Multiline patterns**: whenever your pattern consists of more than one line, use the vertical bar | to indicate a multiline pattern, like in the following Python example: ```yaml pattern: | def myFunction(...): ... ``` * **Metavariables:** metavariables are capital case variables starting with $, like `$N` or `$VAR` . They work as placeholders for actual variables, expressions, or values. They are helpful to write flexible and versatile patterns so that the student is not required to match a piece of code literally. * **Ellipsis operato**r `...` : the `...` ellipsis operator abstracts away a sequence of zero or more items such as arguments, statements, parameters, fields, and characters. * **Literal patterns:** all the parts in your pattern that do not contain metavariables or the `...` operator must be exactly matched by the student's code. Mixing literal and abstract fragments offers a great deal of versatility for devising patterns that can be both as granular and as flexible as desired. ### **Logical operators for rules:** Semgrep allows the creation of composite patterns by applying logical operators on sub-patterns. In CodeGrade, the most commonly used operators are: * `pattern`: a simple single pattern (possibly multiline). * `patterns`: corresponding to the AND logical operator. Other sub-patterns can be nested within it so that the overall pattern will be matched if and only if all the sub-patterns are matched. ```yaml patterns: - pattern: | ... db_query(...) - pattern: ... db_query(..., verify=True, ...) ``` * **`pattern-either`**: corresponding to the OR logical operator. Other sub-patterns can be nested within it so that the overall pattern will be matched if at least one sub-pattern is matched. ```yaml pattern-either: - pattern: hashlib.sha1(...) - pattern: hashlib.md5(...) ``` * `pattern-inside`: The `pattern-inside` operator keeps matched findings that reside within its expression. This is useful for finding code inside other pieces of code, such as functions or blocks. ```yaml patterns: - pattern: return ... - pattern-inside: | class $CLASS: ... - pattern-inside: | def __init__(...): ... ``` ## Semgrep Pattern Collection While our automatic grading guides include examples of Semgrep patterns, we provide a more extensive collection of patterns for the most common programming languages and use-case scenarios here. If you have further questions, don't hesitate to contact us at !
Python ### If statements: * A simple **if** statement whose condition and body can be any: ```yaml pattern: | if ...: ... ``` * A more elaborate **if-else** example. The first **if** condition is that any variable is greater than 5. The **elif** condition is any. The **else** body contains a literal **print** statement that will be matched only if the student prints the same exact string. ```yaml pattern: | if $X > 5: ... elif ...: ... else: print("Number is less than 5") ``` ### Loops * A simple **while** loop. The condition is that a literal variable **x** should be less than any variable. The body can be any. ```yaml pattern: | while x < $VAR: ... ``` * A simple **for** loop. The iterator variable can be any. The body of the loop should contain a statement that prints the iterator. ```yaml pattern: | for $X in range($Y): ... print($X) ... ``` * Check the presence of either a **for** or a **while** loop: ```yaml pattern-either: - pattern: | while ...: ... - pattern: | for ... in ...: ... ``` ### Functions: * function definition, with any signature and body: ```yaml pattern: | def $FUNC(...): ... ``` * function definition where we only care that the first parameter is named **x** and that there is a return statement in the body: ```yaml pattern: | def $F(x, $VAR): ... return ... ``` * a function named **myFunction** is called with the first argument equal to **4** and the second argument any: ```yaml pattern: myFunction(4,$VAR) ``` ### Import statements * if you want to check that a specific object or module is imported, you can just use a literal pattern, like the following: ```yaml pattern: from sklearn.neighbors import KNeighborsClassifier ``` ### Classes * Check that a class named **Dog** is defined**:** ```yaml pattern: | class Dog: ... ``` * Check the presence of the **init method** defined within an arbitrary class. The method should have the following signature: the **self** keyword, a variable named **x,** and a third arbitrary variable. The method's body should contain a literal assignment for an attribute of the class. ```yaml patterns: - pattern-inside: | class $CLASS: ... - pattern: | def __init__(self, x, $Y): ... self.x = x ... ``` Note that also the following pattern would achieve the same result: ```yaml pattern: | class $CLASS: ... def __init__(self, x, $Y): ... self.x = x ... ... ```
Java ### If statements: * A simple **if** statement whose condition and body can be any: ```yaml pattern: | if (...) ... ``` * A more elaborate **if-else** example. The first **if** condition is that any variable is greater than 5. The **else if** condition is any. The **else** body contains a literal **println** statement that will be matched only if the student prints the same exact string. ```yaml pattern: | if ($X > 5) ... else if (...) ... else System.out.println("The number is less than 5."); ``` ### Loops * A simple **while** loop. The condition is that a literal variable **x** should be less than any variable or value. The body can be any. ```yaml pattern: | while (x < $VAR) ... ``` * A simple **for** loop. The iterator is the literal variable **i**. The loop condition and update can be any. The body of the loop should contain a statement that prints a string that must be matched exactly. ```yaml pattern: | for (int i = $VAL; ...; ...) { ... System.out.println("Counting up: " + i); ... } ``` * Check the presence of either a **for** or a **while** loop: ```yaml pattern-either: - pattern: | while (...) ... - pattern: | for (...; ...; ...) ... ``` ### Functions: * function definition, with **public** visibility, return type **String**, and any signature and body: ```yaml pattern: | public String $FUNC(...) { ... } ``` * **public static** function definition that should return a **double**. We check that the first parameter is named **x** and that there is a return statement in the body: ```yaml pattern: | public static double $FUNC(int x, ...) { ... return $VAL; } ``` * a function named **myFunction** is called with the first argument equal to **4** and the second argument any: ```yaml pattern: myFunction(4,$VAR); ``` ### Import statements * if you want to check that a specific object or module is imported, you can just use a literal pattern, like the following: ```yaml pattern: import java.util.ArrayList; ``` ### Classes * Check that a class named **Car** is defined**:** ```yaml pattern: | public class Car{ ... } ``` * Check the presence of specific attributes and methods within the definition of a class named **Car.** We check for: * the declaration of two literal **String** instance attributes, named **make** and **model;** * the declaration of a public Constructor that accepts two **String** parameters. ```yaml patterns: - pattern-inside: public class Car{ ... } - pattern: | ... private String make; - pattern: | ... private String model; - pattern: | public Car (String $PARAM1, String $PARAM2) { ... } ```
C++ ### If statements: * A simple **if** statement whose condition and body can be any: ```yaml pattern: | if ($COND) { ... } ``` * A more elaborate **if-else** example. The first condition can be any. The **else if** condition is that some variable is less than zero. The **else** body prints a message to the screen and must be matched literally. ```yaml pattern: | if ($COND1) { ... } else if ($COND2 < 0) { ... } else { std::cout << "The number is zero." << std::endl; } ``` ### Loops * simple **while** loop. The condition is that some variable should be less than a second one. The body can be any. 
pattern: |
    while($VAR < $VALUE)
      {
        ...
      }
* A simple **for** loop. The loop initialization, condition, and update can be any. The body of the loop can be any too. ```yaml pattern: | for (...; ...; ...) { ... } ``` * Check the presence of an if statement within a while loop: ```yaml patterns: - pattern-inside: | while (...) { ... } - pattern: | if($COND) { ... } ``` ### Functions: * definition of a function where we specify the return type and the types of the input parameters. The function name and body can be any, as long as the latter ends with a return statement. ```yaml pattern: | double $FUNC(int $A, int $B) { ... return $VAL; } ``` * **static** function definition that should return a **double**. We check that the first parameter is of type **int** and is named **x,** and that there is a return statement at the end of the body: ```yaml pattern: | static double $FUNC(int x, ...) { ... return $VAL; } ``` * a function named **myFunction** is called with the first argument equal to **4** and the second argument any: ```yaml pattern: myFunction(4,$VAR); ``` ### Include statements * if you want to check that a specific library is included, you can use a literal pattern, like the following ```yaml pattern: | #include ``` ### Classes Semgrep has not fully supported checks for C++ classes yet. This means that the usual flexibility in designing patterns is not available in this case. Contact us via **** with a description of the check you want to implement, and we will do our best to devise an effective workaround!
C# ### If statements: * A simple **if** statement whose condition and body can be any: ```yaml pattern: | if (...) { ... } ``` * A more elaborate **if-else** example. The first **if** condition is that some variable is greater than 0, and its body is any. The **else if** condition is that the same variable as before is less than 0, and its body is any again. The **else** condition body contains a **Console.WriteLine** statement, which may print anything on the screen. ```yaml pattern: | if ($N > 0) { ... } else if($N < 0) { ... } else { Console.WriteLine(...); } ``` ### Loops * A simple **while** loop. The condition is that some variable, represented by **$VAR**, should be less than 10. The body of the loop can be any, as long as it ends with a post-increment of **$VAR .** ```yaml pattern: | while($VAR<=10) { ... $VAR++; } ``` * The most general **for** loop structure ```yaml pattern: | for (...; ...; ...) { ... } ``` * Check the presence of either a **for** or a **while** loop: ```yaml pattern-either: - pattern: | while (...) { ... } - pattern: | for (...; ...; ...) { ... } ``` ### Functions: * definition of a **static** function that should return an integer value. For the function signature, we specify that it should have two parameters, both of type **int.** The body of the function can be any, but it must terminate with a **return** statement. ```yaml pattern: | static int $F(int $A, int $B) { ... return $VAL } ``` * In the following example, we check that a function named **myFunction** has the following properties: * it returns a **double**; * its first parameter is a **double** variable named **x**. The rest of its signature can be any; * it contains an **if** statement, whose condition is any, and whose body must contain a **return** statement. ```yaml patterns: - pattern-inside: | double myFunction(double x, ...) { ... } - pattern: | if(...) { ... return $VAL } ``` * a function named **myFunction** is called with the first argument equal to **4** and the second argument any: ```yaml pattern: myFunction(4,$VAR); ``` ### Import statements * if you want to check that a specific library is imported, you can just use a literal pattern, like the following: ```yaml pattern: using System; ``` ### Classes * Check that a public class named **Car** is defined**:** ```yaml pattern: | public class Car{ ... } ``` * Check the presence of specific attributes and methods within the definition of a class named **Car.** We check for: * the declaration of two literal **string** data fields, named **make** and **model;** * the declaration of a public function named **DisplayInfo** with return type **void**. ```yaml patterns: - pattern-inside: public class Car { ... } - pattern: | ... private string make; - pattern: | ... private string model; - pattern: | public void DisplayInfo() { ... } ```
R ### If statements: * A simple **if** statement whose condition and body can be any: ```yaml pattern: | if (...) { ... } ``` * A more elaborate **if-else** example. The first **if** condition is that some variable is greater than 0, and its body is any. The **else if** condition is that the same variable as before is less than 0, and its body must contain a **print** statement. The **else** condition body contains a literal **print** statement. ```yaml pattern: | if ($VAR>0) { ... } else if ($VAR<0) { print(...) } else{ print("The variable is zero.") } ``` ### Loops * A simple **while** loop. The condition is that some variable, represented by **$VAR**, should be less than 100. The body of the loop can be any, as long as it ends with an increment of the variable represented by $VAR by one. ```yaml pattern: | while ($VAR < 100) { ... $VAR <- $VAR + 1 } ``` * A **for** loop structure where the iterator variable can be any, and its range goes to some number up to 5. The body of the loop can be any. ```yaml pattern: | for ($VAR in $NUM:5) { ... } ``` * Check the presence of either a **for** or a **while** loop: ```yaml pattern-either: - pattern: | while (...) { ... } - pattern: | for ($VAR in $N:$M) { ... } ``` ### Functions: * Definition of a function that must be named **add\_numbers.** It takes any two parameters as input, and its body can be any as long as it ends with a **return** statement. ```yaml pattern: | add_numbers <- function($A, $B) { ... return($RESULT) } ``` * In the following example, we check that the student defined a function, whose name can be any, with the following properties: * its first parameter is a **double** variable named **x**. The rest of its signature can be any; * it contains an **if** statement, whose condition is any, and whose body must contain a **return** statement. ```yaml patterns: - pattern-inside: | $FUN <- function(x, ...) { ... } - pattern: | if(...) { ... return $VAL } ``` * a function named **myFunction** is called with the first argument equal to **4** and the second argument any: ```yaml pattern: myFunction(4,$VAR) ``` ### Import statements * if you want to check that a specific library is imported, you can just use a literal pattern, like the following: ```yaml pattern: library(ggplot2) ```
# Getting started Learn how to get the most out of CodeGrade as a student! CodeGrade is made to give you the best feedback as quickly as possible. Find out how you can get the most out of CodeGrade and learn how to use some of the tools that your teacher may ask you to use. ## :rocket: Let's get started and hand in our first submission, in which platform do you work? {% content-ref url="/pages/-MLxFU00hf2vAd\_20xOf" %} [Getting started in Blackboard](/for-students/getting-started/getting-started-in-blackboard) {% endcontent-ref %} {% content-ref url="/pages/-MLxFTpWO1cXtV6O-jp\_" %} [Getting started in Brightspace](/for-students/getting-started/getting-started-in-brightspace) {% endcontent-ref %} {% content-ref url="/pages/-MLxFSLDdPt1fqtiiqPt" %} [Getting started in Canvas](/for-students/getting-started/getting-started-in-canvas) {% endcontent-ref %} {% content-ref url="/pages/-MLxFQpzmD7XoVu0pc6p" %} [Getting started in Moodle](/for-students/getting-started/getting-started-in-moodle) {% endcontent-ref %} ## :mortar\_board: Not using any of these platforms? Learn how to use CodeGrade without them! {% content-ref url="/pages/-MLxFRhCQqQXsilMI3-S" %} [Getting started in CodeGrade](/for-students/getting-started/getting-started-in-codegrade) {% endcontent-ref %} {% hint style="success" %} Below, you will find [**Advanced Features**](/for-students/advanced-features) in CodeGrade. These will not be turned on for every assignment and may only be available for you once in a while. {% endhint %} ## :busts\_in\_silhouette: Handing in as a group Some CodeGrade assignments will allow or need you to join a group and upload to them as that group. Learn more about joining groups and handing in for your group here: {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} ## :jigsaw: Handing in using GitHub or GitLab Some CodeGrade assignments will allow you to connect your GitHub or GitLab repository to that CodeGrade assignment, so that you will automatically hand in every time you do a `git push`. {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} ## :pencil: Giving Peer Feedback Some assignments in CodeGrade will need you to review and give feedback to your peers after the deadline has ended. Learn how to do that in the guide below: {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} ## :trophy: Doing a Final Exam in CodeGrade Your teachers can also give final code exams inside CodeGrade, this is nice for you as the workflow will be very similar to the other CodeGrade assignments you had. Learn how to join these exams using the specials links and how final exams work in the guide below: {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ## :question: Asking questions about your code Most CodeGrade assignments have student questions turned on, meaning that you can ask questions about the code to your teachers inside CodeGrade. {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} ## :credit\_card: Enrolling in a paid CodeGrade course Some institutions offer paid courses in CodeGrade, in the guide below you will learn step by step how you can securely pay to enroll in such a course. {% content-ref url="/pages/-MfDazF7VoErv5MwHsTK" %} [Enrolling in a paid course](/for-students/advanced-features/enrolling-in-a-paid-course) {% endcontent-ref %} # Getting started in CodeGrade Learn how to get around in CodeGrade and hand in your first submission! Using CodeGrade as a student is easy and offers you the ability to get a lot more feedback way quicker. When you open an assignment in CodeGrade you are presented with a couple of buttons, which are enabled depending on the settings of your teacher.

Student interface in CodeGrade

An overview of all possible buttons is found below: * **Latest submission**: Go to your latest submission. * **Online Editor:** *(Only available if turned on for your assignment)* * **Continue Editing:** Resume last editor session * **New from template:** Create new submission from template * **New from latest submission:** Create new submisssion from last submission * **Other methods**: *(Only available if turned on for your assignment)* * **Upload files:** Upload files to create a submission. * **Connect Git**: Show instructions on how to connect your assignment to GitHub or GitLab, so that you can automatically upload every time you do a `git push`. * **Assignment description:** Display assignment description (if one is set up). * **Rubric**: Show the rubric - a fancy name for a grading scheme - of this assignment. *(Only available if your assignment has a* [*rubric*](/setup-assignment/build-assignment/setting-up-a-rubric)*)* * **Peer feedback**: Give feedback to your fellow students. (Only available after the deadline and if [peer feedback](/setup-assignment/other-features/setting-up-peer-feedback) is turned on) * **Course feedback**: Show an overview of the feedback you received to all assignments of this course. * **Groups**: Create, join or view groups. *(Only available if your assignment is a* [*group assignment*](/setup-assignment/other-features/making-a-group-assignment)*)* {% hint style="info" %} Some of these are not available when they are not applicable, e.g. when an assignment is not set up as a group assignment, the groups button will be hidden. Please contact your teacher if a button is missing for you. {% endhint %} ## :door: Logging in to CodeGrade You can access CodeGrade by navigating to the link that was shared by your teacher, this is most likely [app.codegra.de](https://app.codegra.de). Here, click on the name of your **institution** to log in to your CodeGrade account for that institution. After logging in, you will find an overview of all courses you are enrolled in and the assignments for these courses. ## :pencil2: Handing in 1. Go to the assignment you want to hand in for via CodeGrade. 2. Select your method of submitting, you can: 1. Open the editor to write and submit your code directly in the browser via the CodeGrade editor. 2. Upload files from your computer to create a submission. 3. Connect your Git repository to upload directly from there. 3. If uploading files: you can either drag and drop files onto the upload field or select them via your browser’s file picker dialog. You can either upload separate files, or an archive (such as a `.zip` or `.tar`). Archives are automatically extracted / unzipped, meaning the individual files and folders in your archive will be your actual submission (but archives contained in other archives are not). 4. Press **"Hand in"**. 5. If your instructor has set up automatic tests, you will now find instant feedback on your submitted code. 6. Click on individual tests to see if they succeeded or failed and why, try to improve your code and hand in again! #### :question: Have a question about your code? {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} #### :jigsaw: Handing in with Git {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} #### :busts\_in\_silhouette: Handing in with a group {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} #### :pencil: Giving Peer Feedback {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} #### :trophy: Doing a Final Exam {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ### :eyes: Viewing feedback After your assignment is graded, you can view your feedback through CodeGrade. 1. Navigate to the assignment in CodeGrade and click on **"Latest Submission"**. 2. View your feedback. On the Feedback Overview, you can view your inline comments with some context. Browse to the Code to view the inline feedback with all of your code. Finally, on the AutoTest tab you can view the output of the Automated Tests. 3. Sometimes, you will receive automated feedback on your style within the code. If a line of code is highlighted, hover over the line number to find the corresponding feedback. If you have given Peer Feedback, a list of your given feedback will be visible in the "*Peer Feedback*" tab. Received feedback from your peers will be displayed as inline comments, just as the feedback from your teacher would be. # Getting started in Blackboard Learn how to get around in CodeGrade for Blackboard and hand in your first submission! Using CodeGrade in Blackboard as a student is easy and offers you the ability to get a lot more feedback way quicker. When you open an assignment in Blackboard that is a CodeGrade assignment, you are presented with a couple of buttons, which are enabled depending on the settings of your teacher.

Student interface in CodeGrade

An overview of all possible buttons is found below: * **Latest submission**: Go to your latest submission. * **Online Editor:** *(Only available if turned on for your assignment)* * **Continue Editing:** Resume last editor session * **New from template:** Create new submission from template * **New from latest submission:** Create new submisssion from last submission * **Other methods**: *(Only available if turned on for your assignment)* * **Upload files:** Upload files to create a submission. * **Connect Git**: Show instructions on how to connect your assignment to GitHub or GitLab, so that you can automatically upload every time you do a `git push`. * **Assignment description:** Display assignment description (if one is set up). * **Rubric**: Show the rubric - a fancy name for a grading scheme - of this assignment. *(Only available if your assignment has a* [*rubric*](/setup-assignment/build-assignment/setting-up-a-rubric)*)* * **Peer feedback**: Give feedback to your fellow students. (Only available after the deadline and if [peer feedback](/setup-assignment/other-features/setting-up-peer-feedback) is turned on) * **Course feedback**: Show an overview of the feedback you received to all assignments of this course. * **Groups**: Create, join or view groups. *(Only available if your assignment is a* [*group assignment*](/setup-assignment/other-features/making-a-group-assignment)*)* {% hint style="info" %} Some of these are not available when they are not applicable, e.g. when an assignment is not set up as a group assignment, the groups button will be hidden. Please contact your teacher if a button is missing for you. {% endhint %} ## :door: Logging in to CodeGrade Accessing CodeGrade is really easy and is done automatically through **Blackboard**. Inside Blackboard, you can simply navigate to your programming assignment as per usual. Clicking on the assignment will present you the CodeGrade interface inside Blackboard, to which you will be automatically logged in. {% hint style="success" %} **Trouble logging in?** CodeGrade supports Chrome, Firefox, Edge and Safari and requires *third party cookies* to be turned on in your browser. Try accessing CodeGrade through another browser before contacting your teacher. {% endhint %} ## :pencil2: Handing in 1. Go to the assignment you want to hand in for via CodeGrade. 2. Select your method of submitting, you can: 1. Open the editor to write and submit your code directly in the browser via the CodeGrade editor. 2. Upload files from your computer to create a submission. 3. Connect your Git repository to upload directly from there. 3. If uploading files: you can either drag and drop files onto the upload field or select them via your browser’s file picker dialog. You can either upload separate files, or an archive (such as a `.zip` or `.tar`). Archives are automatically extracted / unzipped, meaning the individual files and folders in your archive will be your actual submission (but archives contained in other archives are not). 4. Press **"Hand in"**. 5. If your instructor has set up automatic tests, you will now find instant feedback on your submitted code. 6. Click on individual tests to see if they succeeded or failed and why, try to improve your code and hand in again! #### :question: Have a question about your code? {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} #### :jigsaw: Handing in with Git {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} #### :busts\_in\_silhouette: Handing in with a group {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} #### :pencil: Giving Peer Feedback {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} #### :trophy: Doing a Final Exam {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ### :eyes: Viewing feedback After your assignment is graded, you can view your feedback through CodeGrade. 1. Navigate to the assignment in CodeGrade and click on **"Latest Submission"**. 2. View your feedback. On the Feedback Overview, you can view your inline comments with some context. Browse to the Code to view the inline feedback with all of your code. Finally, on the AutoTest tab you can view the output of the Automated Tests. 3. Sometimes, you will receive automated feedback on your style within the code. If a line of code is highlighted, hover over the line number to find the corresponding feedback. If you have given Peer Feedback, a list of your given feedback will be visible in the "*Peer Feedback*" tab. Received feedback from your peers will be displayed as inline comments, just as the feedback from your teacher would be. # Getting started in Brightspace Learn how to get around in CodeGrade for D2L Brightspace and hand in your first submission! Using CodeGrade in D2L Brightspace as a student is easy and offers you the ability to get a lot more feedback way quicker. When you open an assignment in D2L Brightspace that is a CodeGrade assignment, you are presented with a couple of buttons, which are enabled depending on the settings of your teacher.

Student interface in CodeGrade

An overview of all possible buttons is found below: * **Latest submission**: Go to your latest submission. * **Online Editor:** *(Only available if turned on for your assignment)* * **Continue Editing:** Resume last editor session * **New from template:** Create new submission from template * **New from latest submission:** Create new submisssion from last submission * **Other methods**: *(Only available if turned on for your assignment)* * **Upload files:** Upload files to create a submission. * **Connect Git**: Show instructions on how to connect your assignment to GitHub or GitLab, so that you can automatically upload every time you do a `git push`. * **Assignment description:** Display assignment description (if one is set up). * **Rubric**: Show the rubric - a fancy name for a grading scheme - of this assignment. *(Only available if your assignment has a* [*rubric*](/setup-assignment/build-assignment/setting-up-a-rubric)*)* * **Peer feedback**: Give feedback to your fellow students. (Only available after the deadline and if [peer feedback](/setup-assignment/other-features/setting-up-peer-feedback) is turned on) * **Course feedback**: Show an overview of the feedback you received to all assignments of this course. * **Groups**: Create, join or view groups. *(Only available if your assignment is a* [*group assignment*](/setup-assignment/other-features/making-a-group-assignment)*)* {% hint style="info" %} Some of these are not available when they are not applicable, e.g. when an assignment is not set up as a group assignment, the groups button will be hidden. Please contact your teacher if a button is missing for you. {% endhint %} ## :door: Logging in to CodeGrade Accessing CodeGrade is really easy and is done automatically through **Brightspace**. Inside Brightspace, you can simply navigate to your programming assignment as per usual. Clicking on the assignment will present you the CodeGrade interface inside Brightspace, to which you will be automatically logged in. {% hint style="success" %} **Trouble logging in?** CodeGrade supports Chrome, Firefox, Edge and Safari and requires *third party cookies* to be turned on in your browser. Try accessing CodeGrade through another browser before contacting your teacher. {% endhint %} ## :pencil2: Handing in 1. Go to the assignment you want to hand in for via CodeGrade. 2. Select your method of submitting, you can: 1. Open the editor to write and submit your code directly in the browser via the CodeGrade editor. 2. Upload files from your computer to create a submission. 3. Connect your Git repository to upload directly from there. 3. If uploading files: you can either drag and drop files onto the upload field or select them via your browser’s file picker dialog. You can either upload separate files, or an archive (such as a `.zip` or `.tar`). Archives are automatically extracted / unzipped, meaning the individual files and folders in your archive will be your actual submission (but archives contained in other archives are not). 4. Press **"Hand in"**. 5. If your instructor has set up automatic tests, you will now find instant feedback on your submitted code. 6. Click on individual tests to see if they succeeded or failed and why, try to improve your code and hand in again! #### :question: Have a question about your code? {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} #### :jigsaw: Handing in with Git {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} #### :busts\_in\_silhouette: Handing in with a group {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} #### :pencil: Giving Peer Feedback {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} #### :trophy: Doing a Final Exam {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ### :eyes: Viewing feedback After your assignment is graded, you can view your feedback through CodeGrade. 1. Navigate to the assignment in CodeGrade and click on **"Latest Submission"**. 2. View your feedback. On the Feedback Overview, you can view your inline comments with some context. Browse to the Code to view the inline feedback with all of your code. Finally, on the AutoTest tab you can view the output of the Automated Tests. 3. Sometimes, you will receive automated feedback on your style within the code. If a line of code is highlighted, hover over the line number to find the corresponding feedback. If you have given Peer Feedback, a list of your given feedback will be visible in the "*Peer Feedback*" tab. Received feedback from your peers will be displayed as inline comments, just as the feedback from your teacher would be. # Getting started in Canvas Learn how to get around in CodeGrade for Instructure Canvas and hand in your first submission! Using CodeGrade in Instructure Canvas as a student is easy and offers you the ability to get a lot more feedback way quicker. When you open an assignment in Canvas that is a CodeGrade assignment, you are presented with a couple of buttons, which are enabled depending on the settings of your teacher.

Student interface in CodeGrade

An overview of all possible buttons is found below: * **Latest submission**: Go to your latest submission. * **Online Editor:** *(Only available if turned on for your assignment)* * **Continue Editing:** Resume last editor session * **New from template:** Create new submission from template * **New from latest submission:** Create new submisssion from last submission * **Other methods**: *(Only available if turned on for your assignment)* * **Upload files:** Upload files to create a submission. * **Connect Git**: Show instructions on how to connect your assignment to GitHub or GitLab, so that you can automatically upload every time you do a `git push`. * **Assignment description:** Display assignment description (if one is set up). * **Rubric**: Show the rubric - a fancy name for a grading scheme - of this assignment. *(Only available if your assignment has a* [*rubric*](/setup-assignment/build-assignment/setting-up-a-rubric)*)* * **Peer feedback**: Give feedback to your fellow students. (Only available after the deadline and if [peer feedback](/setup-assignment/other-features/setting-up-peer-feedback) is turned on) * **Course feedback**: Show an overview of the feedback you received to all assignments of this course. * **Groups**: Create, join or view groups. *(Only available if your assignment is a* [*group assignment*](/setup-assignment/other-features/making-a-group-assignment)*)* {% hint style="info" %} Some of these are not available when they are not applicable, e.g. when an assignment is not set up as a group assignment, the groups button will be hidden. Please contact your teacher if a button is missing for you. {% endhint %} ## :door: Logging in to CodeGrade Accessing CodeGrade is really easy and is done automatically through **Canvas**. Inside Canvas, you can simply navigate to your programming assignment as per usual. Clicking on the assignment will present you the CodeGrade interface inside Canvas, to which you will be automatically logged in. {% hint style="success" %} **Trouble logging in?** CodeGrade supports Chrome, Firefox, Edge and Safari and requires *third party cookies* to be turned on in your browser. Try accessing CodeGrade through another browser before contacting your teacher. {% endhint %} ## :pencil2: Handing in 1. Go to the assignment you want to hand in for via CodeGrade. 2. Select your method of submitting, you can: 1. Open the editor to write and submit your code directly in the browser via the CodeGrade editor. 2. Upload files from your computer to create a submission. 3. Connect your Git repository to upload directly from there. 3. If uploading files: you can either drag and drop files onto the upload field or select them via your browser’s file picker dialog. You can either upload separate files, or an archive (such as a `.zip` or `.tar`). Archives are automatically extracted / unzipped, meaning the individual files and folders in your archive will be your actual submission (but archives contained in other archives are not). 4. Press **"Hand in"**. 5. If your instructor has set up automatic tests, you will now find instant feedback on your submitted code. 6. Click on individual tests to see if they succeeded or failed and why, try to improve your code and hand in again! #### :question: Have a question about your code? {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} #### :jigsaw: Handing in with Git {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} #### :busts\_in\_silhouette: Handing in with a group {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} #### :pencil: Giving Peer Feedback {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} #### :trophy: Doing a Final Exam {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ### :eyes: Viewing feedback After your assignment is graded, you can view your feedback through CodeGrade. 1. Navigate to the assignment in CodeGrade and click on **"Latest Submission"**. 2. View your feedback. On the Feedback Overview, you can view your inline comments with some context. Browse to the Code to view the inline feedback with all of your code. Finally, on the AutoTest tab you can view the output of the Automated Tests. 3. Sometimes, you will receive automated feedback on your style within the code. If a line of code is highlighted, hover over the line number to find the corresponding feedback. If you have given Peer Feedback, a list of your given feedback will be visible in the "*Peer Feedback*" tab. Received feedback from your peers will be displayed as inline comments, just as the feedback from your teacher would be. # Getting started in Moodle Learn how to get around in CodeGrade for Moodle and hand in your first submission! Using CodeGrade in Moodle as a student is easy and offers you the ability to get a lot more feedback way quicker. When you open an assignment in Moodle that is a CodeGrade assignment, you are presented with a couple of buttons, which are enabled depending on the settings of your teacher.

Student interface in CodeGrade

An overview of all possible buttons is found below: * **Latest submission**: Go to your latest submission. * **Online Editor:** *(Only available if turned on for your assignment)* * **Continue Editing:** Resume last editor session * **New from template:** Create new submission from template * **New from latest submission:** Create new submisssion from last submission * **Other methods**: *(Only available if turned on for your assignment)* * **Upload files:** Upload files to create a submission. * **Connect Git**: Show instructions on how to connect your assignment to GitHub or GitLab, so that you can automatically upload every time you do a `git push`. * **Assignment description:** Display assignment description (if one is set up). * **Rubric**: Show the rubric - a fancy name for a grading scheme - of this assignment. *(Only available if your assignment has a* [*rubric*](/setup-assignment/build-assignment/setting-up-a-rubric)*)* * **Peer feedback**: Give feedback to your fellow students. (Only available after the deadline and if [peer feedback](/setup-assignment/other-features/setting-up-peer-feedback) is turned on) * **Course feedback**: Show an overview of the feedback you received to all assignments of this course. * **Groups**: Create, join or view groups. *(Only available if your assignment is a* [*group assignment*](/setup-assignment/other-features/making-a-group-assignment)*)* {% hint style="info" %} Some of these are not available when they are not applicable, e.g. when an assignment is not set up as a group assignment, the groups button will be hidden. Please contact your teacher if a button is missing for you. {% endhint %} ## :door: Logging in to CodeGrade Accessing CodeGrade is really easy and is done automatically through **Moodle**. Inside Moodle, you can simply navigate to your programming assignment as per usual. Clicking on the assignment will present you the CodeGrade interface inside Moodle, to which you will be automatically logged in. {% hint style="success" %} **Trouble logging in?** CodeGrade supports Chrome, Firefox, Edge and Safari and requires *third party cookies* to be turned on in your browser. Try accessing CodeGrade through another browser before contacting your teacher. {% endhint %} ## :pencil2: Handing in 1. Go to the assignment you want to hand in for via CodeGrade. 2. Select your method of submitting, you can: 1. Open the editor to write and submit your code directly in the browser via the CodeGrade editor. 2. Upload files from your computer to create a submission. 3. Connect your Git repository to upload directly from there. 3. If uploading files: you can either drag and drop files onto the upload field or select them via your browser’s file picker dialog. You can either upload separate files, or an archive (such as a `.zip` or `.tar`). Archives are automatically extracted / unzipped, meaning the individual files and folders in your archive will be your actual submission (but archives contained in other archives are not). 4. Press **"Hand in"**. 5. If your instructor has set up automatic tests, you will now find instant feedback on your submitted code. 6. Click on individual tests to see if they succeeded or failed and why, try to improve your code and hand in again! #### :question: Have a question about your code? {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} #### :jigsaw: Handing in with Git {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} #### :busts\_in\_silhouette: Handing in with a group {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} #### :pencil: Giving Peer Feedback {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} #### :trophy: Doing a Final Exam {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ### :eyes: Viewing feedback After your assignment is graded, you can view your feedback through CodeGrade. 1. Navigate to the assignment in CodeGrade and click on **"Latest Submission"**. 2. View your feedback. On the Feedback Overview, you can view your inline comments with some context. Browse to the Code to view the inline feedback with all of your code. Finally, on the AutoTest tab you can view the output of the Automated Tests. 3. Sometimes, you will receive automated feedback on your style within the code. If a line of code is highlighted, hover over the line number to find the corresponding feedback. If you have given Peer Feedback, a list of your given feedback will be visible in the "*Peer Feedback*" tab. Received feedback from your peers will be displayed as inline comments, just as the feedback from your teacher would be. # I forgot my CodeGrade username / password Learn how you can recover your username or password in case you forgot them! {% hint style="warning" %} This guide only applies to you when you use CodeGrade as a stand alone tool. Your account management is done in your LMS if you use CodeGrade through Canvas, Blackboard, Brightspace, Moodle, Sakai or Open edX! {% endhint %} #### Follow the steps below to recover your CodeGrade username: Your CodeGrade username was probably manually set when registering your account or enrolling to your course. Luckily, we have sent you an email with your username to confirm your registration. Please find this email and also check your spam folder. If you cannot find this email anymore, please reach out to your instructor, who should be able to find a full list with all usernames in their course in CodeGrade. #### Follow the steps below to recover your CodeGrade password: To recover your CodeGrade password, you do need to know your username. If you do, navigate to CodeGrade and press the "**Forgot password**" link. ![Click the "Forgot password" link.](/files/OLsO8Ai1V0tPOJbYZGDk) Afterwards, fill in your CodeGrade username (please note, this is a username that could be different from your LMS username or student email address). And press **Request email**. ![Fill in your username and press "Request email".](/files/bU4gkkMewZTCjjyByUGX) You will now get an email to the email address linked to your CodeGrade account, which includes a link that will allow you to set up a new password for your account. {% hint style="success" %} CodeGrade recommends you to generate a secure password for your account and use one of the many secure password managers to store it. {% endhint %} #### **You are now ready to start using CodeGrade again and handing in to your assignments! Learn how to use CodeGrade below:** :point\_down: {% content-ref url="/pages/-MLxFRhCQqQXsilMI3-S" %} [Getting started in CodeGrade](/for-students/getting-started/getting-started-in-codegrade) {% endcontent-ref %} # Advanced Features Find out about all things you can do as a student in CodeGrade, from connecting your Git repo to asking questions and giving peer feedback. {% hint style="success" %} Below, you will find Advanced Features in CodeGrade. These will not be turned on for every assignment and may only be used once in a while. {% endhint %} ## :busts\_in\_silhouette: Handing in as a group Some CodeGrade assignments will allow or need you to join a group and upload to them as that group. Learn more about joining groups and handing in for your group here: {% content-ref url="/pages/-MMKnFa4gLIjG3YhOJua" %} [Handing in with a group](/for-students/advanced-features/handing-in-as-a-group) {% endcontent-ref %} ## :jigsaw: Handing in using GitHub or GitLab Some CodeGrade assignments will allow you to connect your GitHub or GitLab repository to that CodeGrade assignment, so that you will automatically hand in every time you do a `git push`. {% content-ref url="/pages/-MLxFpdJdDCr4fH5qKOX" %} [Handing in using Git](/for-students/advanced-features/handing-in-using-git) {% endcontent-ref %} ## :pencil: Giving Peer Feedback Some assignments in CodeGrade will need you to review and give feedback to your peers after the deadline has ended. Learn how to do that in the guide below: {% content-ref url="/pages/-MLxFudjAiUASTwenP5Y" %} [Giving Peer Feedback](/for-students/advanced-features/giving-peer-feedback) {% endcontent-ref %} ## :trophy: Doing a Final Exam in CodeGrade Your teachers can also give final code exams inside CodeGrade, this is nice for you as the workflow will be very similar to the other CodeGrade assignments you had. Learn how to join these exams using the specials links and how final exams work in the guide below: {% content-ref url="/pages/-MLxFuVQVFQlvXqqwWS5" %} [Doing a Final Exam](/for-students/advanced-features/doing-a-final-exam) {% endcontent-ref %} ## :question: Asking questions about your code Most CodeGrade assignments have student questions turned on, meaning that you can ask questions about the code to your teachers inside CodeGrade. {% content-ref url="/pages/-MLxJ5Bz1B5Pl\_FeWMi5" %} [Asking Questions](/for-students/advanced-features/asking-questions) {% endcontent-ref %} # Handing in with a group If your CodeGrade assignment is a group assignment, you can make or join a group to hand in for that whole group. If your CodeGrade assignment is a group assignment, you can make or join a group to hand in for that whole group and the other group members can also hand in for the entire group. After handing in your submission, everyone in the group will be able to see this submission and automatic results going along with it. 1. After navigating to the group assignment for the first time, press the "**Groups**" button to join or create a group. 2. After joining a group, you can hand in for this group as usual, all your submissions are now handed in and visible to everyone in your group. {% hint style="success" %} Depending on the configuration of your teacher, you can sometimes only join groups and sometimes also create new groups yourself. Please ask your teacher if you have troubles joining a group. {% endhint %} {% hint style="info" %} For group assignments, you can **only** start handing in when you are in a group with a sufficient number of group members. {% endhint %} #### :woman\_teacher: Learn how to make your CodeGrade assignment into a group assignment: {% content-ref url="/pages/-MMKlImjUnQ5KPfla\_7S" %} [Group Assignments](/setup-assignment/other-features/making-a-group-assignment) {% endcontent-ref %} # Handing in using Git If your teacher has turned on Git uploading, you can upload directly from your GitHub or GitLab repository to CodeGrade. {% hint style="success" %} Some assignments may not have Git uploading turned on, ask your teacher about this if you cannot find this option in CodeGrade. {% endhint %} Some CodeGrade assignments allow you to connect your Git repository, so that every time you do a Git push, your work is uploaded to CodeGrade too. Setting up Git is very straightforward and has to be done only once. #### Step 1: Click on the "Connect Git" button in CodeGrade. Connect Git button. You can only connect your Git repository to CodeGrade if this button is available to you. If it is not, your instructor did not turn Git on. #### Step 2: Select your Git host. ![Select the Git host in CodeGrade.](/files/-Mj50wV4S_Nyk6S3KWHg) Select which Git host you use. Choose GitHub if you use that website and choose GitLab if you use GitLab or if your university has their own Git installation. After selecting the host, you will be taken to their website to log in and authorize CodeGrade. {% hint style="success" %} **Already made a Git connection before?** You only have to authorize CodeGrade once! So if you already made a Git connection before, you can directly select or create a repository for your CodeGrade assignment. {% endhint %} #### Step 3: Log in and authorize CodeGrade. ![Log in to your Git account and authorize CodeGrade.](/files/-Mj519hwkBXiEWJT0Tia) You will only have to log in and authorize CodeGrade once, after that it will be available for all your other assignments inside CodeGrade too. #### Step 4: Select your repo or create a new one. ![Connecting a repository to CodeGrade.](/files/-Mj51MLPfxe248FZpUIz) After logging in and authorizing, you will find an overview of your repositories. You can connect an existing repository or create and connect a new one to this assignment. #### Step 5: Connection made! After selecting a repository, this repository will be cloned to CodeGrade and a first submission will be made. After this, you can start to use Git like you normally would and every `git push` command will result in a new submission in CodeGrade automatically! {% hint style="info" %} **Want to sign out of Git?** You can undo your Git connection with CodeGrade by revoking access to the External Tool "CodeGrade" in your GitHub or GitLab account (this is managed via their website). {% endhint %} ### Manually setting up a Git connection The steps below teach you how you can set up a Git connection manually: 1. Click on the **Set up Git** button, click on "Or manually connect your Git repository.", now select your Git host. 2. Next, copy the *deploy key* from the page to paste it in your repository. Go to the settings of your GitHub/GitLab repository, locate the **Deploy Keys** option and paste the key into the corresponding input field and click on the **Add key** button below it. You can leave the *Write access allowed* checkbox unchecked. 3. Go to the **Add the webhook** step in CodeGrade, and copy the URL. Next, go the webhook settings of your GitHub/GitLab repository, paste the URL into the corresponding input field. Follow the same for **Secret Token** as well. 4. Make sure the *Push events* trigger is turned on and *Enable SSL verification* is checked in your webhook settings. Click on **Add webhook** to connect your repository to CodeGrade. Now, every time you execute Git push, your work is automatically submitted to CodeGrade. 5. Your GitHub or GitLab repository is now connected to CodeGrade. Perform a `git push` to test out if everything is set up correctly. If setting up Git manually, it is a good practice to check whether your webhook connection is set up correctly, simply do a git push if you have work to push or do an empty commit with the following command: `git commit --allow-empty -m "Create a CodeGrade submission" && git push` . #### :woman\_teacher: Learn how to turn on Git uploading for your assignment in CodeGrade: {% content-ref url="/pages/F0zZ1aj7XLlwCqbDZYOz" %} [General settings](/setup-assignment/build-assignment/general-settings) {% endcontent-ref %} # Giving Peer Feedback In some assignments, you are asked to give feedback to the code of your peers. Some assignments in CodeGrade are designed to allow for peer feedback, where you can give your peers constructive comments on their work and review their code. Follow the steps below to learn how to give peer feedback on your peers: 1. Once the deadline has passed, you will find the **Peer feedback** button enabled next to the disabled **Upload files** button. 2. Click on the **Peer feedback** button to find the list of students that you need to give feedback to. You can also find the deadline for this at the bottom of the list. 3. Click on the student you want to give feedback to and select the file you want to give feedback in. Click on the line you want to give feedback on, write your comments in the input field and click on the **Save** button to record it. You can also give general feedback to other students, using the General Feedback button (see image below) and saving it with the **Submit** button. ![Giving general feedback to a peer](/files/km8VvyouBTnYcNev0qeX)

Giving general feedback to a peer

Teachers have the option to finally approve or disapprove any of your comments, the feedback you write will then be displayed to your peer. {% hint style="success" %} It can be that your peers uploaded multiple files, be sure to review all files your teacher requested you to review! {% endhint %} {% hint style="success" %} On the right side of the list with students you have to review, it shows the number of comments you have already given to each student. It may contain two numbers when the other student has done another submission after you have already given them some feedback. If this happens, the first number is the amount of comments you have given on their latest submission, and the second is the total amount of comments you placed on all of their submissions (the last one included). {% endhint %} #### :woman\_teacher: Learn how to turn on peer feedback for your assignment: {% content-ref url="/pages/-MMKlIaLtrLi8WydKWVb" %} [Peer Feedback](/setup-assignment/other-features/setting-up-peer-feedback) {% endcontent-ref %} # Doing a Final Exam Final exams in CodeGrade give you a very similar experience as regular assignments, with the difference of how and when you can access it. Follow the steps below to join your exam in CodeGrade: 1. For most exams in CodeGrade, you will receive an email with a private link to access the exam. Click on the link to automatically get logged into CodeGrade and arrive on the exam page. 2. This page displays the start and finish time of the exam, and also how much time is left until you can start the exam. 3. Once the exam time begins, click on the **Start** button to start handing in to CodeGrade. 4. Once you enter the exam, you only have access to the course of the exam. Click on **Upload files** to submit your work, as you normally do in CodeGrade. {% hint style="success" %} If you did not receive an email with a private link, chances are that you can simply access your exam through the CodeGrade app directly. {% endhint %} # Asking Questions As a student, you can ask your teachers questions for some of your CodeGrade assignments. For some courses, you will be able to ask your teachers questions about your code, automatic results and the assignment directly from within CodeGrade. Just as the feedback you get from your teachers, you can ask your questions directly about the line or section of code it is about. 1. After handing in your submission, click on the "**Latest Submission**" button to display your code. 2. Click on any line of your code to start typing your question that is about that line or section of code. 3. After writing your question, press "**Save**" to send it to your teacher. {% hint style="success" %} Have a more general question about the code that does not apply to just one section? Simply press the first line of code to ask this. {% endhint %} --- [Next Page](/llms-full.txt/1)