Skip to content
Docs

Exams

GroutCode Coding Exams

The code_spec format, starter and test files, and encrypted hidden tests with key custody.

A coding exam is an exam with a code_spec. GroutCode installs the toolchain locally, opens the brief, runs sample tests on demand and, when the student asks, fetches the hidden-test key once per attempt so hidden tests can run offline without ever shipping the key inside the exam.

code_spec#

json
{
  "v": 1,
  "brief_md": "# Implement a stack\n…",
  "toolchain": "python",
  "starter": { "kind": "zip", "file": { "id": "file_…" } },
  "sample_tests": { "kind": "inline", "files": [{ "path": "tests/sample/test_basic.py", "content": "…" }] },
  "hidden_tests": { "kind": "zip-encrypted", "file": { "id": "file_…" }, "iv_b64": "…", "sha256": "…" },
  "commands": { "setup": "", "run": "python main.py", "test_sample": "pytest {{workspace}}/tests/sample", "test_hidden": "pytest {{hidden}}", "timeout_ms": 60000 },
  "ai_policy": { "allowed": false, "model_tier": "any" },
  "rubric_md": "…",
  "exclude_globs": ["node_modules/**", ".venv/**"]
}
FieldNotes
toolchainc, cpp, java, python, node, sqlite. Missing toolchains are installed on the student's machine.
starter{kind:"zip", file} uploaded via POST /exams/{id}/files, or {kind:"empty"}.
sample_testsInline files (relative paths, no ..) or a zip. Visible to the student.
hidden_testsAES-256-GCM-encrypted zip. iv_b64 and the ciphertext sha256 live in the spec; the key does not.
commands{{workspace}}, {{hidden}}, {{results}} are substituted. timeout_ms 1 000–600 000.
ai_policyEnforced locally; usage is reported at submit in ai_usage_data.

Hidden-test workflow#

  1. 1

    Encrypt and upload

    Zip the hidden tests, encrypt with AES-256-GCM using a random 12-byte IV, and upload the ciphertext:

    bash
    curl -X POST $API/exams/ex_…/files -H "Authorization: Bearer $KEY" -F "[email protected]"

    Put the returned ref in code_spec.hidden_tests.file, plus iv_b64 and the ciphertext sha256.

  2. 2

    Store the key

    bash
    curl -X PUT $API/exams/ex_…/code-secret -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{ "hidden_key_b64": "<base64 of 32 random bytes>" }'

    The key is write-only for students; GroutCode obtains it through a release call that stamps the attempt.

  3. 3

    Watch releases

    bash
    curl $API/exams/ex_…/code-secret/status -H "Authorization: Bearer $KEY"
curl $API/exams/ex_…/attempts -H "Authorization: Bearer $KEY"        # hidden_released_at per student
  4. 4

    Re-grade offline (optional)

    POST /exams/{id}/code-secret/release returns the key to an institution integration so you can decrypt and re-run hidden tests yourself.

Encrypting in practice#

js
import { randomBytes, createCipheriv, createHash } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';

const key = randomBytes(32), iv = randomBytes(12);
const cipher = createCipheriv('aes-256-gcm', key, iv);
const ct = Buffer.concat([cipher.update(readFileSync('hidden-tests.zip')), cipher.final(), cipher.getAuthTag()]);
writeFileSync('hidden-tests.zip.enc', ct);
console.log({ hidden_key_b64: key.toString('base64'), iv_b64: iv.toString('base64'), sha256: createHash('sha256').update(ct).digest('hex') });

What comes back at submit#

GroutCode uploads workspace.zip, transcript.json (AI transcript) and test-results.json. They appear as signed attachment URLs on GET /submissions/{id}, and ai_usage_data carries the local test results, timing, focus-loss and models used:

json
{ "kind": "groutcode", "v": 1,
  "tests": { "sample": { "passed": 8, "failed": 0 }, "hidden": { "passed": 11, "failed": 1, "hidden_zip_sha256": "…", "workspace_sha256": "…" } },
  "timing": { "started_at": "…", "hidden_released_at": "…", "submitted_at": "…" },
  "focus": { "lost_count": 2, "lost_seconds": 41 },
  "toolchain": "python", "app": { "name": "groutcode", "version": "2.4.0", "platform": "win32" },
  "total_interactions": 0, "models_used": [] }
PreviousExams OverviewNextSubmissions & Grading