ai assessment

Generating Question Banks for Coding With AI

EduGenius Team··19 min read

Watch the EduGenius tutorials playlist

Feature walkthroughs, setup help, and practical learning workflows connected to this article.

Open Tutorials

Generating Question Banks for Coding With AI

A coding question bank needs one real addition beyond the standard five tags, and it isn't an external-asset tag the way music needed one for notation images and recordings. A code snippet is self-contained text — nothing about it depends on a file that can go offline. The genuine addition is an item-type tag (predict-output, spot-the-bug, explain-in-plain-English, reorder-the-code), plus a small wrinkle on content standard: naming the language and version alongside the concept.

Quick Answer: A coding question bank inherits the standard five tags — content standard, Bloom's level, format, difficulty, last-used — with one wrinkle (content standard needs a language/version qualifier) and one real addition (item type). It does not need a music-style external-dependency tag with a durability field, because a snippet's correctness, unlike a lost recording, can always be re-checked for free by running it in a live interpreter.

The origin guide in this series already drew coding's hard line: a static, AI-generated document has no interpreter behind it, so it can score a selected or short constructed response, never an executed one. The true/false guide built on that line to find a documented misconception — students expect b = a to copy a list, and it doesn't.

Neither guide asked what a standing bank, assembled across a whole school year, should tag by. This guide does — and it tests the obvious candidate, borrowed from music's bank guide, against a concrete retrieval scenario before trusting it.

This guide covers:

  • What a coding bank inherits unmodified, and the one wrinkle content standard picks up
  • Why music's external-asset tag doesn't transfer to coding, tested directly rather than assumed
  • The real second axis a coding bank needs, and why it isn't the same shape as music's or science's
  • A full tag set, a step-by-step build workflow, and what's known about AI's tagging accuracy

This entry sits inside the same subject-by-subject series as How to Create Spanish Quizzes With AI, where each guide tests a shared format question against one subject's real structure.

What a Coding Question Bank Inherits From General Bank Architecture

A coding bank starts from the same five tags any Bloom's-aligned bank needs, and most of them need no coding-specific rework. The Bloom's-aligned bank pillar set that baseline against a living table-of-specifications blueprint: content standard, Bloom's level, item format, difficulty, and last-used or exposure (Notar, Zuelke, Wilson, & Yunker, 2004).

Four Tags That Transfer Without Modification

Bloom's level, item format, difficulty, and last-used work the same for a coding item as for any other subject in this series. A loop-tracing item sits at Apply or Analyze; it's MCQ or short-answer; it's easy or hard; it's been seen or it hasn't.

  • Bloom's level and difficulty don't discriminate between coding's own item types any more than they discriminate between a biology recall item and an inference item.
  • Item format (MCQ, true/false, short-answer) carries over cleanly, but — as the next section shows — it can't do the sorting work a coding bank actually needs.
  • Last-used/exposure works identically to any subject: it tracks whether a student has already seen an item, nothing coding-specific about it.

The Wrinkle: Content Standard Needs a Language, Not Just a Concept

A concept like "loops" or "functions" is language-agnostic in principle, but a specific item never is — it's written in Python, JavaScript, or block-based pseudocode, and its stated "correct" answer can depend on which one. That's a real wrinkle on content standard, the same shape the science bank guide found when a cross-domain item needed a paired value instead of one.

Two items can share an identical concept tag — "integer division" — while one is true in Python 3 and false in Python 2, and a bare concept tag with no language or version attached can't tell a teacher which item is which.

Large-scale programs already solve this by pinning one language outright. The College Board's AP Computer Science A exam is written and scored entirely in Java, with a fixed, published subset of methods and classes students can rely on (College Board, 2024) — a real-world precedent for treating language identity as load-bearing, not incidental, metadata.

Testing the Obvious Candidate: Does Coding Need an Asset-Dependency Tag?

The obvious next move copies music's bank guide directly: music needed a stimulus-asset tag for notation images and an audio-source tag for recordings, because neither dependency was nameable from content standard alone. Before assuming coding needs the same thing, it's worth running music's own test against a coding item instead of just borrowing the conclusion.

Does a Snippet Depend on an External Asset the Way a Staff Image Does?

Music's tag exists because a notation item's correctness is inseparable from one specific, sourced image, and an ear-training item's correctness is inseparable from one specific recording — neither is something AI can generate or a document can carry inline. A code snippet fails that test immediately.

  • A snippet is plain text, fully contained inside the question stem, exactly like the rest of the item.
  • Nothing about it points outside the document the way a "see attached staff excerpt" instruction does.
  • There is no equivalent of a lost recording. A snippet doesn't reference a file that a subscription or a shared drive could stop serving.

So the direct parallel fails on its own terms: coding has no analog to a staff image or an audio clip. That's a real answer, not a shortcut past the question — the two subjects' dependency structures are genuinely different, not just differently named.

The Real Candidate: Language Version and Runtime Behavior

The prompt worth taking seriously is narrower: does a snippet's correctness depend on something external the way music's audio depends on a subscription staying live? Language version is the honest candidate. Python's integer division changed behavior between Python 2 and Python 3 — / used to truncate, now it returns a float, with // added for floor division. A library API can get deprecated the same way, quietly changing what "correct" output looks like for identical code.

That's a real risk. An item bank's own quality-assurance literature treats this kind of drift as expected across any domain: item pools need continuous monitoring to stay healthy, not a one-time check at creation (von Davier et al., 2026).

Where the Parallel Actually Breaks

Here's the difference that settles whether coding needs music's kind of tag, not just a version of it. The true/false guide's closing advice already names the fix for version drift: run every snippet through a real interpreter before publishing. That fix has no equivalent in music.

PropertyMusic's Audio DependencyCoding's Version Dependency
Can the original be regenerated for free?No — AI cannot generate real playable audioYes — any language/version has a free, stable interpreter
What happens if the source goes stale?The clip may be permanently unrecoverableThe snippet can be re-run and re-verified in minutes
What kind of tag does it justify?An external-dependency tag with a durability fieldA periodic re-verification habit, not a durability tracker

A stale audio link can cost an item permanently. A stale version assumption costs a five-minute re-run. That gap in recoverability, not a difference in how "real" the risk feels, is why coding doesn't need music's tag shape.

The Real Second Axis: Item Type, Not Asset Dependency

If content standard's language wrinkle handles version risk, and the asset-dependency parallel doesn't transfer, the actual gap left in a coding bank is a different one entirely: nothing yet records which of coding's four item types a given item is. The origin guide already established that predict-output, spot-the-bug, explain-in-plain-English, and reorder-the-code are the four item types a static document can honestly support. A bank has to be able to retrieve by that split.

Retrieval Test: Pulling Every Exact-Match-Gradable Item

Say a bank holds 180 items across a school year, and a teacher wants to build a self-checking practice station — one that grades itself with no human review — ahead of a substitute-covered period. That needs every item gradable by exact string comparison: predict-output and reorder-the-code, never explain-in-plain-English.

Content standard doesn't answer this; "loops" holds all four item types about loops with nothing to tell them apart. Format doesn't either, once checked against what a coding bank actually holds:

  • MCQ can be a predict-output item, a selected-response spot-the-bug item, or a reorder item — three different item types, one format tag.
  • Short-answer can be a constructed spot-the-bug explanation or an explain-in-plain-English item — both need a rubric, but they test different skills.
  • True/false, per the true/false guide, compresses output-prediction cleanly but only a narrowed slice of explain — same format tag, different underlying item type.

Retrieval Test: Pulling Every Item Tied to One Language Before an Update

A second, independent need: a department switches a unit from Python 2-style examples to Python 3, or upgrades a JavaScript unit past a deprecated method, and a teacher needs every item written against the old version before the term starts. That's the content-standard wrinkle from the section above doing its job — language and version, tracked as part of content standard, not item type.

These two retrieval needs are genuinely independent. An output-prediction item and a spot-the-bug item can both be written in the same language version; a Python-2-era item and a current-Python item can both be predict-output items. Neither tag substitutes for the other.

Why Grading Method Doesn't Need a Third Tag

It's tempting to add a separate "autogradable: yes/no" tag on top of item type. That's redundant once item type and format are both recorded, the same move the science bank guide made when it skipped a third "full skill vs. isolated slice" tag because two existing tags already implied it.

  1. Predict-output and reorder-the-code are exact-match gradable by definition — no format changes that.
  2. Explain-in-plain-English always needs a rubric — again, true regardless of format.
  3. Spot-the-bug is the one item type where format decides it: selected-response ("which line") is exact-match; constructed-response ("what's wrong, and why") needs a rubric.

Item type plus format already answer "can this be autograded" for every item in the bank. A third tag would only restate information the first two already carry.

The Tag Set a Coding Question Bank Actually Needs

A workable coding bank keeps the general five tags, lets content standard carry a language/version qualifier, and adds one new tag: item type. That's a lighter addition than music's two new tags, and a differently-shaped one than science's single convergence tag.

TagCarries Over Unmodified?Notes for Coding
Content standardYes, with a language/version qualifier"Python 3: loops," not just "loops" — version matters when output can differ across versions
Bloom's levelYesDoesn't discriminate between item types; tag it anyway for the reasons any bank needs it
Item formatYesMCQ, true/false, short-answer; each format can hold more than one item type, so it can't sort alone
DifficultyYesEasy/medium/hard, or a tracked pass rate, unchanged
Last used/exposureYesSame purpose as any bank, any subject
Item type (new)NoPredict-output, spot-the-bug (selected or constructed), explain-in-plain-English, reorder-the-code

Coding does not get a stimulus-asset or audio-source-style tag. A snippet carries no external dependency a bank needs to track the way music's notation and audio items do.

A Grade-Calibrated Worked Example

Say you teach a grade 6 Python class finishing a unit on loops and conditionals. Four items from that unit land on four different item-type values, all tagged "Python 3: loops":

  • Predict-output. A loop summing scores above a threshold, asking what prints at the end — exact-match gradable, MCQ or short-answer.
  • Spot-the-bug, selected. The same loop with the accumulator variable never initialized, asking which line fails — exact-match, MCQ.
  • Spot-the-bug, constructed. The identical broken loop, asking a student to name the missing line and explain why — rubric-scored.
  • Reorder-the-code. The working version, scrambled into five lines, asking a student to restore the sequence — exact-match.

Tagged only by content standard and format, a pull for "Python 3: loops, MCQ" would return the predict-output item and the selected spot-the-bug item mixed together, indistinguishable without opening each one. Item type separates them in one filter.

Building a Coding Question Bank: A Step-by-Step Workflow

Building a coding bank follows the general bank workflow, with one extra decision made the moment any item is written: naming its item type and language/version before it's filed.

  1. Set a standing blueprint with concept down one axis and item type across the other (Notar et al., 2004), leaving room for a language/version note in each cell.
  2. Pick the item type before drafting — predict-output, spot-the-bug, explain, or reorder — since it decides whether exact-match grading or a rubric applies.
  3. Name the language and version explicitly in the prompt and the tag, especially for anything touching division, string concatenation, or a library method that has changed behavior across versions.
  4. For predict-output and reorder items, confirm the output is deterministic — skip floating-point results, unordered collections, and anything version-sensitive unless the item is scoped tightly enough to be safe.
  5. For spot-the-bug, decide selected or constructed up front, since that choice is what actually determines whether the item is autogradable.
  6. Run every snippet through a real interpreter before filing it, not just before the first use — the same check that resolves version drift instead of a tracked durability field.
  7. Tag every item on all six fields at creation. Retagging a backlog later costs far more than tagging while the decision is still fresh.
  8. Re-verify version-sensitive items when a curriculum shifts language or version, using the content-standard tag to pull the affected set in one query.

You could use EduGenius to hold a class profile's grade level and current language/version while drafting a batch of predict-output, spot-the-bug, and reorder items in one pass, then export the set with an answer key once every snippet checks out in a real interpreter.

Can AI Tag These Fields Accurately?

No dedicated accuracy research exists yet for classifying coding items by item type, the same honest gap the science bank guide found for its own new tag and the music bank guide found for stimulus-asset and audio-source. The tagging-speed guide reports real published figures for automated Bloom's-level classification specifically — roughly 72-73% for general-purpose AI tools, rising to the low-to-mid 90s for purpose-built classifiers (Kumar, Gulwani, & Singh, 2025) — but that evidence doesn't extend to item-type classification.

That gap matters less for a new item than it sounds. A writer already decides whether a snippet is being traced, debugged, explained, or reordered before it's written at all; the tag is a byproduct of that decision, not a separate research problem.

Tools for Building a Coding Question Bank

No mainstream AI content tool ships with an item-type field the way some flag Bloom's level automatically; the real difference is how reliably a tool holds a stated language/version and item type across a whole drafting session.

ToolTags Item Type at CreationHolds Language/Version ContextAutograde-Ready Output (predict-output, reorder)Starting Price
EduGeniusIf stated in the promptYes, via saved class profilesYes, when requested25 welcome credits free; Starter $7.99/month (500 credits); Professional $15.99/month (1,000 credits)
General-purpose AI chat toolsOnly if explicitly requestedNo persistent memory by defaultOnly if explicitly requestedVaries by provider
Execution-based coding platforms (e.g., Codewars, HackerRank)N/A — grades by running submitted code, not by drafting bank itemsN/AN/AVaries by platform
Spreadsheet or LMS native item banksFully manual; teacher enters every fieldDepends on platformDepends on how it's builtOften bundled with an existing LMS, or free

No tool automatically confirms a snippet's stated output is actually correct — that check still belongs to a teacher, run against a real interpreter, before the item reaches a bank.

Pro Tips for a Coding Question Bank

  • Tag item type before writing the item, not after — it determines whether the item needs a deterministic output check or a rubric.
  • Name the language and version in every prompt, even when it feels obvious. "Python" alone doesn't say whether / truncates or returns a float.
  • Skip a "gradable" tag entirely. Item type plus format already answer it, the same way the science bank guide found a third tag redundant with two existing ones.
  • Re-run version-sensitive items whenever a curriculum shifts language or version, using the content-standard tag to pull the affected set instead of re-reading the whole bank.
  • Reserve reorder-the-code items for working, verified snippets. The coding worksheets guide covers building a Parsons-style scramble as low-stakes practice before it becomes a graded bank item.

What to Avoid When Building a Coding Question Bank

Copying Music's External-Asset Tag Without Testing It

Music needed a stimulus-asset and an audio-source tag because notation and ear-training items depend on files AI can't regenerate. A code snippet has no such dependency — borrowing the tag shape without testing it against coding's actual structure adds a field with nothing real to track.

Treating Language/Version as Optional Metadata

A concept tag like "loops" without a language and version attached can silently mix a Python-2-era item with a current one, or hide a genuinely different-behavior JavaScript item under the same label. Name the version every time output could differ across one.

Assuming a Snippet's Stated Output Is Correct Without Re-Running It

AI-generated programming content is fluent, not automatically correct. A 2024 study of 517 real programming questions and their ChatGPT-generated answers found 52% contained incorrect information (Kabir, Udo-Imeh, Kou, & Zhang, 2024) — a real error rate worth taking seriously before an item goes into a bank at all, let alone stays there for years.

Building a "Durability" Field Coding Doesn't Need

Music's audio-source tag needed an owned-versus-streamed field because a lost recording is often unrecoverable. Adding that same field to a coding bank manufactures a risk category that doesn't exist — a snippet is never one dead link away from being unverifiable.

Key Takeaways

  • A coding question bank needs one real new tag: item type — predict-output, spot-the-bug, explain-in-plain-English, or reorder-the-code — not an external-asset tag like music's.
  • Content standard picks up a real wrinkle: a language and version qualifier, since identical-looking code can behave differently across Python 2 and 3, or across a deprecated library API.
  • The direct music parallel fails on its own terms. A snippet is self-contained text with no equivalent of a staff image or a recording that can go permanently offline.
  • The recoverability gap is what actually settles it: a stale audio link can cost an item forever; a stale version assumption costs a five-minute re-run in a free interpreter.
  • Item type and format together already answer whether an item is autogradable, so a third "gradable" tag would only restate information two existing tags already carry.
  • Tagging item type and language/version at creation is nearly free, since both fall out of decisions a writer already makes while drafting the item.
  • No published accuracy research yet covers automated item-type classification for coding, the same honest gap this series found for science's and music's own new tags.

Frequently Asked Questions

Does a coding question bank need a tag for external files, the way a music question bank needs one for audio and notation?

No. A code snippet is self-contained text with no equivalent of a staff image or an audio recording — nothing about it points to an external file that could go offline. The real new tag a coding bank needs is item type: predict-output, spot-the-bug, explain-in-plain-English, or reorder-the-code.

Can a coding question bank item become wrong over time?

Yes, if it depends on version-specific behavior — Python's integer division changed between Python 2 and 3, and library APIs get deprecated. Unlike a lost audio file, though, a stale coding item can always be re-verified for free by running the snippet in a current interpreter.

What's the most important tag to add to a general question bank for coding?

Item type. Content standard and item format both fail to separate a predict-output item from a spot-the-bug item from an explain item, tested against a real retrieval scenario — pulling every exact-match-gradable item for a self-checking practice station.

How is organizing a coding question bank different from organizing a music or science one?

Music's bank guide needed two new tags because notation and ear-training items depend on external assets AI can't regenerate. Science's bank guide needed one tag because its two candidate axes converged at one content type. Coding needs one new tag too, but for a different reason: its four item types are the axis that actually varies independently of content and format, with no asset-dependency risk to track at all.

Sources

  • Notar, C. E., Zuelke, D. C., Wilson, J. D., & Yunker, B. D. (2004). The table of specification: Insuring accountability in teacher made tests. Journal of Instructional Psychology, 31(2), 115-129.
  • Ihantola, P., Ahoniemi, T., Karavirta, V., & Seppälä, O. (2010). Review of recent systems for automatic assessment of programming assignments. Proceedings of the 10th Koli Calling International Conference on Computing Education Research, 86-93.
  • Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024). Is Stack Overflow obsolete? An empirical study of the characteristics of ChatGPT answers to Stack Overflow questions. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems.
  • Kumar, R., Gulwani, D., & Singh, S. (2025). Automated analysis of learning outcomes and exam questions based on Bloom's taxonomy. arXiv preprint arXiv:2511.10903.
  • von Davier, A. A., Zhang, X., Attali, Y., Park, Y., Church, J., Runge, A., LaFlair, G. T., & Tsigler, A. (2026). Analytics for quality assurance for item pools (AQuAP): Monitoring and maintaining item bank health in AI-driven assessment systems. arXiv preprint arXiv:2606.18536.
  • College Board. (2024). AP Computer Science A Course and Exam Description.
  • Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.
#teachers#assessment#quiz