🎨Automatically grading CSS

In this page we describe how to automatically test css rules for web development assignments.

While in principle it is possible to use Selenium to test the css properties defined for html elements, we advise against doing so in CodeGrade. The reason is that we run Selenium tests by using a web browser in headless mode, which can result in css rules working differently than they would while running your web browser with a normal display. Alternatively we can use Semgrep to test the structure of the css rules defined by the student.

Design Semgrep patterns for testing Css rules

Semgrep is used to perform code analysis by writing syntax rules in a human readable format. Rules are made by generic or language specific patterns, which are then searched for in the student's code. With its pattern syntax, you can find:

• Equivalences: Matching code that means the same thing even though it looks different;

• Wildcards / ellipsis (...): Matching any statement, expression or variable;

• Metavariables ($X): Matching unknown expressions that you do not yet know what they will exactly look like, but want to be the same variable in multiple parts of your pattern;

• Logical operators : it is possible to logically combine simpler patterns to create more complex aggregate patterns.

Let's inspect an example of a Semgrep pattern that checks the properties of a btn css class:

patterns:
      - pattern-inside: |
         .btn { ... }
      - pattern: |
         ...
         margin: 20px;
         ...
      - pattern: |
          ...
          width: $ANYVALUE;
          ...

• Line 1: with patterns we indicate that our overall pattern is composed of multiple sub-patterns;

• Line 2 to 3: we require that the patterns which will be defined later have to be found within the body of a class named btn;

• Line 4 to 7: we require that our pattern contains a margin setting of exactly 20 pixels;

• Line 8 to 11: we require that our pattern contains a width setting with any possible value (notice the use of a metavariable)

We suggest using the online Semgrep editor to quickly develop new rules for testing css rules.

Setup

CodeGrade AutoTest V2 runs on Ubuntu (20.04 LTS) machines which you can configure in any way that you want. Common software is pre-installed, but you can also manually install software.

In the setup section of your AutoTest, you can install software or packages and upload any files you might need for testing. The setup section will build an image and is only run once for all submissions.

Step 1

First we need to install Semgrep : drag and drop a Script Block where you can type the following bash command:

python3 -m pip install semgrep

Step 2

The second step is upload the python script (which will be built-in in a later version). structure.py that we use to run Semgrep .

2KB

structure.py

AutoTest V2 Setup configuration to use Semgrep.

Test Phase

To run a Semgrep test in CodeGrade:

1. Drag an IO Test block to your test configuration. Execute the following script in the code field: python3 $UPLOADED_FILES/structure.py <STUDENTFILE>. For testing a css file, this could be: python3 $UPLOADED_FILES/structure.py styles.css.

2. Create and drag one or more Full match block(s) inside your IO Test block. Each block is one Semgrep structure to check. As Input write the Semgrep pattern (including pattern: or similar like pattern-either:).

3. As Expected Output write "Found!" if the student code should match or "Not found!" if the student code should not contain the pattern (e.g. students are not allowed to use numpy).

4. Optionally: Use a "Connect Rubric" block to connect your structure check to a rubric category.

5. Optionally: Use a "Hide" block to hide the "config" (i.e. input / pattern) of your tests so that students cannot see the pattern you are checking for.

A Semgrep test for Css rules. The test configuration is hidden to the students and the test is connect to a rubric category.