ai assessment

Generating Formative Assessments for Coding With AI

EduGenius Team··17 min read

Watch the EduGenius tutorials playlist

Feature walkthroughs, setup help, and practical learning workflows connected to this article.

Open Tutorials

Generating Formative Assessments for Coding With AI

A 60-second coding check works best when it asks a student to trace a value, not write one — but not every item on the coding quiz guide's list of four valid types is equally cheap to run mid-lesson. Selected-response predict-output and reorder-the-code items grade themselves instantly; explain-in-plain-English and open-ended spot-the-bug items still need a human or rubric pass, which is exactly the overhead a fast formative check exists to avoid.

There's a second wrinkle underneath the first: tracing a loop and tracing a conditional aren't the same cognitive act, and treating them as interchangeable "predict-output" items hides real information about where a class's understanding actually breaks.

The coding quiz guide settled the big question already: a static, AI-generated document has no interpreter behind it, so it can score a selected answer or a short constructed response, but it can never compile or run an original function a student submits. That wall doesn't move for a formative check. What moves is everything downstream of it — which of the four valid item types actually survives a 60-second, ungraded, mid-lesson version of the same idea.

Quick Answer: The hub's four item types — predict-output, spot-the-bug, explain, reorder — all remain technically valid for a formative check. But formative's real constraint is response format, not item-type label: selected-response versions of predict-output and reorder grade themselves and fit a short cycle; constructed-response explain and open spot-the-bug items still cost rubric time. Inside predict-output specifically, tracing a loop is a harder, more diagnostic act than tracing a conditional, and that difference is worth building into how a formative sequence is ordered.

This guide covers:

  • Why dropping the score doesn't remove the coding quiz guide's execution wall, only its distractor-building overhead
  • The real formative-specific finding: response format, not item type, decides what fits a short cycle
  • Why tracing a loop and tracing a conditional are genuinely different checks, not two flavors of the same one
  • A two-step formative sequence built around that distinction
  • A step-by-step AI workflow, a tools comparison, and the mistakes that undercut a fast coding check

Coding's formative check joins a growing set of subject-specific breakdowns on this site, including Generating Formative Assessments for Biology With AI and Generating Formative Assessments for Geography With AI — each one asking the same underlying question about a different subject's own structural constraint.

What Dropping the Score Changes, and What It Doesn't

A formative coding check inherits the quiz guide's execution wall unchanged — no AI tool can run a student's original function — but it sheds the distractor-construction work a graded quiz item requires. That's a real difference in effort, not a difference in what's possible.

What the Quiz Guide Had to Solve

Building a scored multiple-choice item means writing wrong answers that fail for a specific, nameable reason: a skipped loop condition, a misread operator, an off-by-one error. The quiz guide's output-prediction table shows exactly this — four answer choices, each tied to one plausible misreading. That work takes real time, and it's worth it for a graded instrument a student sees once.

What a Formative Check Skips

Researcher D. Royce Sadler's foundational definition frames formative assessment's job as closing the gap between where a student's understanding currently is and where instruction assumes it is (Sadler, 1989). A formative check doesn't need a defensible wrong-answer set to do that job — it needs a fast, honest read of what a student actually produces, right or wrong.

  • A quiz item's job: build a traceable wrong answer for every distractor, defensible enough to survive a grade appeal
  • A formative check's job: ask the same underlying question, ungraded, and read whatever comes back as a signal to act on

The Timing Constraint Is the Real Driver

Dylan Wiliam and Marnie Thompson describe formative assessment's defining feature as operating on a short cycle — minute-by-minute, inside a single lesson, rather than day-by-day or unit-by-unit (Wiliam & Thompson, 2007). That timing window, not any change in what a static document can score, is what actually filters which of the hub's four item types earn their keep in a formative setting.

The Real Formative-Specific Finding: Response Format Beats Item-Type Label

All four of the coding quiz guide's item types remain valid for a formative check, but they don't survive a short cycle equally — the deciding factor is whether a response is selected or constructed, not which of the four categories it falls into. That's a genuine sub-distinction the quiz guide didn't need to resolve, since a graded quiz has time to build a rubric either way.

Selected-Response Items Are Formative-Native

An output-prediction MCQ or a "which line is wrong" spot-the-bug item grades itself the instant a student answers, whether that's a raised hand, a whiteboard, or a polling tool. A reorder-the-code item works the same way once it's built: a scrambled snippet has exactly one correct sequence, so checking it is a comparison, not a judgment call (Parsons & Haden, 2006).

This isn't a hypothetical advantage. Peer Instruction, a technique built around fast, in-class multiple-choice questions followed by student discussion, has an unusually large evidence base specifically inside introductory programming courses. Researchers Leo Porter and Beth Simon documented measurable gains in course retention using exactly this format across multiple CS1 sections (Porter & Simon, 2013).

  • Output-prediction MCQ: self-grading, works on a show-of-hands or a polling tool
  • Spot-the-bug, "which line": self-grading, same mechanics as output-prediction
  • Reorder-the-code: self-grading once the scrambled snippet is built, one correct sequence to check against

Constructed Items Still Cost Rubric Time

Explain-in-plain-English items and open-ended spot-the-bug items ("what's wrong, and why?") don't self-grade, because more than one correctly-worded answer can be right. That's not a flaw in the item type — the quiz guide's rubric section already covers building a partial-credit rubric for exactly this reason. It's a mismatch with a 60-second window, where a teacher can't run even a quick rubric pass on 25 written answers before the lesson moves on.

A constructed-response explain item isn't wrong for formative use — it's better suited to an exit ticket a teacher reads after class than to a check meant to redirect the next five minutes of instruction.

Where the Four Types Land for Formative Use

Item TypeResponse FormatSelf-Grading?Formative Fit
Output-predictionSelected (MCQ)YesStrong — near-zero overhead, works live
Spot-the-bugSelected ("which line")YesStrong — same mechanics as output-prediction
Spot-the-bugConstructed ("what's wrong, why")NoWeak for in-lesson use — better as a post-class exit check
Reorder-the-codeSelected (one correct order)YesStrong — quick once the snippet is built
Explain-in-plain-EnglishConstructedNoWeak for in-lesson use — pair with a short rubric read later

None of this contradicts the quiz guide's four-type boundary. It sharpens it: the label "predict-output" or "explain" was never the variable that mattered for a scored quiz, where rubric time is already budgeted in. For a formative check running inside a five-minute window, response format is the variable that decides whether the check survives contact with the clock.

Inside Predict-Output: Why a Loop Trace Isn't a Conditional Trace

Tracing a loop and tracing a conditional both fall under "predict-output" in the quiz guide's taxonomy, but they ask a student to do genuinely different mental work, and collapsing them into one formative item type hides where a class's real gap sits. This is the sub-distinction the hub's four-way split doesn't surface on its own.

The State-Tracking Difference

A conditional evaluates once: a student checks a single condition against a single state and picks a branch. A loop asks a student to hold a running, mutating value across repeated passes — what researchers call the program's notional machine, the mental model of what the computer is doing step by step (du Boulay, 1986; Sorva, 2013).

# Conditional trace — one decision, one state
grade = 78
if grade >= 80:
    result = "pass"
else:
    result = "retry"
print(result)
# Loop trace — state updates across repeated passes
total = 0
for n in [4, 7, 2, 9]:
    total += n
print(total)

Both snippets are short. Only the second asks a student to keep a running value in mind across four separate updates before answering.

What the Research Says About Difficulty

A widely-cited study of novice programmers by Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Järvinen found that students consistently rated loop structures as more difficult than selection structures, even at comparable code length (Lahtinen, Ala-Mutka, & Järvinen, 2005). The reason lines up with the notional-machine explanation: a loop's control variable updates automatically, "behind the scene," in a way a novice has to visualize rather than read off the page directly.

  • Conditional trace: one decision, evaluated once, closer to a recall-and-apply task
  • Loop trace: a mutating value tracked across multiple passes, closer to a sustained working-memory task
  • Same surface label ("predict-output"), genuinely different cognitive demand underneath

What the Skill Hierarchy Adds

A separate line of research gives this distinction more weight for formative sequencing specifically. Raymond Lister and colleagues found evidence of a skill hierarchy in novice programmers: reading code, tracing it, explaining it, and writing it form roughly ordered stages, where weakness in an earlier stage predicts weakness later (Lopez, Whalley, Robbins, & Lister, 2008). An earlier analysis from the same research program found that students who traced code correctly less than half the time typically could not explain equivalent code either (Philpott, Robbins, & Whalley, 2007).

If tracing ability is a leading indicator for explaining ability, a fast trace-check that catches a gap early is cheaper — in both class time and student frustration — than discovering the same gap later through a constructed-response explain item that also costs rubric time to grade.

SOLO Taxonomy Maps the Same Split

The quiz guide already applies the SOLO taxonomy to separate syntax-recall items from logic-comprehension items (Biggs & Collis, 1982). The loop-versus-conditional split fits neatly inside that same framework, one level apart.

SOLO LevelTrace TypeWhat a Correct Answer Requires
MultistructuralSingle conditionalOne condition checked against one state
RelationalLoop with accumulationA value tracked and updated across several passes
Relational, denserLoop containing a conditionalBoth skills at once — the state-tracking and the branch

A quiz that only ever traces conditionals plateaus at the multistructural level no matter how many items it includes. A formative sequence that never asks a loop-trace question never actually tests whether a class's mental model of iteration is working.

A Two-Step Formative Sequence

Given the difficulty gap, a practical formative sequence runs the cheaper check first and reserves the loop-trace item to confirm the deeper model.

  1. Open with a conditional-trace MCQ. Fast to answer, fast to write distractors for, and a reasonable warm-up signal that basic branching logic has landed.
  2. Follow with a loop-trace MCQ on related code. If most of the class clears the conditional check but stumbles here, that's a specific, actionable signal: the gap is in iteration, not in branching.
  3. Branch instruction on the result, not on a single combined score: strong on both → move on; strong on conditionals only → slow down on loops specifically; weak on both → reteach branching before touching iteration at all.

A Grade-Calibrated Illustration

Say you teach a grade 4 class working in a block-based tool and want a 60-second check on an "if score is greater than 10, say Great Job" block. That's a conditional trace using block syntax, appropriate before a class has touched typed conditions at all.

A grade 7 Python class that has just covered for loops is ready for the denser version: a four-item list, a running total, and a one-line prediction question. Same underlying skill category as the block-based check, one SOLO level up.

You could use EduGenius to hold a class profile's grade level and current unit while drafting a matched pair of conditional-trace and loop-trace multiple-choice items from a pasted snippet, then display or print both as a quick two-question check once each snippet has been verified in a real interpreter.

A Step-by-Step AI Workflow for Fast Coding Checks

Building a usable fast coding check starts with picking the response format before the topic, since format is what decides whether a check survives a short cycle at all.

  1. Name the exact skill, not just the unit: "tracing a loop with accumulation," not "loops."
  2. Choose selected-response by default. Reserve constructed-response explain or open spot-the-bug items for an exit ticket read after class, not a live mid-lesson check.
  3. Decide conditional or loop deliberately, and don't treat them as interchangeable "predict-output" requests in a prompt.
  4. Supply the real snippet, not a topic name, so distractors tie to a specific misreading instead of a random wrong number.
  5. Set your class profile's grade level and language, block-based pseudocode for younger grades, real syntax once a class works in an actual language.
  6. Anchor grade-band expectations to your standards' algorithms and control-structure strands (CSTA, 2017).
  7. Run every snippet through a real interpreter before use. A generated answer key is a draft, not a verified one, until it's been checked (ISTE, 2025).

Tools Compared for Fast Coding Checks

ToolDistinguishes Selected vs. Constructed ResponseSeparates Loop vs. Conditional Trace ItemsClass-Context MemoryStarting Price
EduGeniusYes, if specified in the promptYes, if requested explicitlyYes, via saved class profiles25 welcome credits free; Starter $7.99/month (500 credits); Professional $15.99/month (1,000 credits)
General-purpose AI chat toolsOnly if explicitly requestedOnly if explicitly requestedNo, unless re-typed each timeVaries by provider
Live-response platforms (polling/clicker tools)Yes, via MCQ or poll fieldsDepends entirely on the uploaded questionYes, within a saved classFree tiers with paid upgrades
Whiteboard or show-of-hands checkYes, inherently selected-responseOnly if the teacher writes it that wayNoNo cost

No tool in that table verifies a generated snippet's stated output is actually correct. A 2024 study of 517 real programming questions and their ChatGPT-generated answers found 52% contained incorrect information (Kabir, Udo-Imeh, Kou, & Zhang, 2024) — a real-enough error rate that a quick run through an actual interpreter stays a non-negotiable step, formative or graded. The general formative-assessment speed guide covers this short-cycle timing problem across every subject; this guide narrows it to coding's own trace-difficulty split.

Pro Tips for Fast Coding Checks

  • Ask "selected or constructed?" before "which item type?" That question decides more about formative fit than the label predict-output, spot-the-bug, or explain ever will.
  • Don't average a conditional-trace score with a loop-trace score into one combined result. Keep them separate so a gap in iteration doesn't get masked by strength in branching.
  • Use a loop-trace item as your confirmation check, not your opening one. It costs more attention to answer, so spend it once basic branching is already confirmed.
  • Borrow the coding worksheet guide's trace-the-output format for ungraded practice before turning the same skill into a formative check.
  • Keep a bank of matched conditional/loop pairs on hand for a unit on control structures, so a mid-lesson check never has to be written from scratch.

What to Avoid

Treating a Formative Check as a Shortcut Around the Execution Wall

Dropping the score doesn't unlock the ability to run a student's original code. A formative check that asks a student to "write a function" and expects instant grading is asking a static tool to do something only an execution-based platform like Codewars or HackerRank can actually do (Ihantola, Ahoniemi, Karavirta, & Seppälä, 2010).

Using a Constructed-Response Explain Item as a Live Mid-Lesson Check

An open-ended "explain what this does" question is genuinely valuable, but reading 25 written answers in real time isn't realistic. Save constructed-response items for an exit ticket you read after the bell, not a check meant to redirect the next five minutes.

Collapsing Loop and Conditional Checks Into One Undifferentiated "Predict Output" Bucket

If a check mixes conditional and loop items without separating the results, a class that's strong on branching but weak on iteration looks like a class with a moderate, unremarkable score — when the real story is a specific, addressable gap.

Trusting a Generated Snippet's Stated Output Without Running It

Fluent code is not the same as correct code. Every snippet used in a live check, formative or graded, needs a run through a real interpreter first, especially given how often AI-generated technical answers contain real errors (Kabir et al., 2024).

Key Takeaways

  • Formative use doesn't loosen the coding quiz guide's execution wall. No AI tool can compile or run a student's original code, scored or not.
  • All four hub item types remain valid for formative use, but they don't survive a short cycle equally. Response format — selected versus constructed — decides formative fit more than the item-type label does.
  • Selected-response predict-output, "which line" spot-the-bug, and reorder-the-code are formative-native, self-grading and fast enough for a live mid-lesson check.
  • Constructed-response explain and open spot-the-bug items still need rubric time, better suited to an exit ticket read after class.
  • Tracing a loop and tracing a conditional are genuinely different checks, not two versions of the same "predict-output" item (Lahtinen, Ala-Mutka, & Järvinen, 2005).
  • A skill hierarchy links tracing ability to explaining ability, making a fast trace-check a cheap, early-warning signal for a gap that would otherwise surface later and cost more to diagnose (Lopez et al., 2008).
  • A two-step sequence — conditional-trace first, loop-trace to confirm — turns that difficulty gap into an actionable classroom routine, not just a research footnote.

Frequently Asked Questions

Can a formative coding check grade a student's own written code?

No. A formative check inherits the same wall a graded coding quiz has: a static, AI-generated document has no interpreter behind it, so it can score a selected answer or a short constructed response, but not compile or run an original function a student submits (Ihantola et al., 2010).

Which coding item type works best for a fast, mid-lesson check?

Selected-response output-prediction and "which line" spot-the-bug items work best, since both grade themselves instantly and need no rubric review. Constructed-response explain items are valuable but fit better as an exit ticket read after class than a live check meant to redirect the next few minutes of instruction.

Is tracing a loop actually harder than tracing a conditional, or does it just look that way?

Research on novice programmers has found students consistently rate loop structures as more difficult than conditionals, even at similar code length, because a loop requires tracking a value that updates automatically across repeated passes rather than evaluating one condition once (Lahtinen, Ala-Mutka, & Järvinen, 2005).

How is this different from the general formative-assessment speed guide on this site?

The general guide covers the short-cycle timing problem across any subject. This guide covers what's specific to coding: a genuine difficulty gap between two subtypes of the same "predict-output" item, and a response-format distinction that decides which of the hub's four item types actually survive a live classroom check.

#teachers#assessment#quiz