ai assessment

Generating Diagnostic Assessments for Geography With AI

EduGenius Team··19 min read

Watch the EduGenius tutorials playlist

Feature walkthroughs, setup help, and practical learning workflows connected to this article.

Open Tutorials

Generating Diagnostic Assessments for Geography With AI

A geography diagnostic assessment is a pre-unit, ungraded check, given before a single lesson on a new region or system this year, that surfaces what a student already believes about how a map represents the world or why a place developed the way it did. Its job isn't measuring whether today's teaching landed. It's finding the belief a lesson would otherwise have to work against without ever noticing it was there.

Quick Answer: Geography does not have a widely validated, psychometrically tested diagnostic instrument built for K-9 content the way biology has the Conceptual Inventory of Natural Selection or physics has the Force Concept Inventory. What it has instead is a set of individually well-documented misconceptions — map-projection size distortion (Lapon, Ooms, & De Maeyer, 2020), continent-country conflation (Catling & Rawlinson, 2014), and everyday-experience physical-geography beliefs (Dove, 1999) — surfaced through the same lightweight, real precedent this series keeps returning to: Angelo and Cross's Background Knowledge Probe (1993).

Say you're about to open a Grade 6 unit on world regions. You've taught it before: continents, countries, the rough shape of where things sit on a map you'll project on day one.

What you don't know, on the Friday before the unit starts, is how many students already believe that map's shapes are simply true — that a country drawn twice as large on the wall really is twice as large in the world. That's a belief baked into every map a student has seen, and you can't correct a distortion nobody's told you is there.

This guide covers:

  • Why a pre-unit geography diagnostic asks a different question than a mid-lesson geography formative check
  • The honest answer to whether geography has its own validated diagnostic instrument, and what the field has instead
  • Three documented geography beliefs worth surfacing before a unit opens, each anchored to a named, real source
  • A step-by-step AI workflow, a tools comparison, pitfalls, and FAQ

This guide sits alongside How to Create Spanish Quizzes With AI in this site's subject-specific series, where each entry looks for a format's genuinely different design problem in that subject rather than reusing one skeleton across every one.

Why a Pre-Unit Diagnostic Asks a Different Question Than a Formative Check

A geography diagnostic and a geography formative check fail for different reasons because they sit on opposite sides of instruction, not at different points on the same clock. The formative-assessment guide for this subject found that geography's fast, 30-90-second in-lesson checks mostly hinge on whether a map is already displayed. A diagnostic runs into a different constraint entirely, because there's no lesson yet for a map to be displayed for.

What a Mid-Lesson Formative Check Gets to Assume

A formative check dropped into a running lesson can assume a map is already on the wall for another reason, or that a concept was just taught this period. It's measuring whether that teaching is landing, in real time, against a backdrop the lesson itself supplied.

What a Pre-Unit Diagnostic Can't Assume

A diagnostic given before a unit's first lesson has no backdrop and no lesson to check against. It's measuring what a student walked in with this year: a prior grade's coverage, a family road trip, a video game's fictional map, a news segment, or nothing at all.

Design QuestionFormative Check (mid-instruction)Diagnostic Assessment (pre-instruction)
TimingDuring the current lesson, often riding on a map already displayedBefore the unit's first lesson this year
What it measuresWhether today's teaching is landingWhat a student already believes, before any teaching
Best prompt shapeA quick map-reading or described "why" checkAn open self-report on an existing belief
Geography example"Which shaded region has the higher population density?" (map already up)"When you look at a world map, do you think the sizes you see are accurate?" (pre-unit)
What a wrong response meansA gap in today's specific instructionA belief the unit needs to actively target

Why the Distinction Isn't Just Semantics

Cognitive scientists Posner, Strike, Hewson, and Gertzog's conceptual-change research made the underlying case decades ago: a misconception has to be surfaced and directly confronted before new instruction can genuinely displace it, or the old belief quietly persists alongside the new material (Posner, Strike, Hewson, & Gertzog, 1982). A teacher who never learns a belief about map size or continent identity exists can't design a unit that corrects it.

That timing gap matters more in geography than it might first appear. A 2016 U.S. Government Accountability Office review found that roughly three-quarters of eighth-grade students were not rated "proficient" in geography, unchanged since 1994 (U.S. Government Accountability Office, 2016). Instruction that never checks what a class walks in with is fighting that gap blind.

Does Geography Have Its Own Validated Diagnostic Instrument?

No — unlike biology's Conceptual Inventory of Natural Selection or physics's Force Concept Inventory, there is no widely validated, psychometrically tested diagnostic instrument built specifically for K-9 geography's actual content mix of map skills, physical systems, and human systems. That's the honest starting point for this guide, and it's worth saying plainly rather than inventing a substitute that sounds more rigorous than it is.

What Biology and Physics Have That Geography Doesn't

The biology diagnostic guide in this series covers biology's real, published instrument: the Conceptual Inventory of Natural Selection, a 20-item multiple-choice test whose wrong-answer options are built from documented student misconceptions, validated through years of interviews and statistical testing (Anderson, Fisher, & Norman, 2002). Physics has its own equivalent in the Force Concept Inventory. Neither instrument has a direct geography counterpart.

  • A validated concept inventory needs years of interview-based item development plus large-scale statistical testing before it can be trusted as a diagnostic, not just a quiz.
  • Geography's content is also more fragmented than a single-topic biology unit: map literacy, physical systems, and human systems don't compress into one 20-item instrument the way natural selection does.
  • No comparable large-scale, K-9-focused instrument has emerged from geography education research to fill that gap.

The Geoscience Concept Inventory Is a Near Miss, Not a Match

The closest real precedent sits in an adjacent field, not geography itself. Researchers Julie Libarkin and Steven Anderson built the Geoscience Concept Inventory, a validated multiple-choice instrument piloted with thousands of students, designed to catch entry-level college misconceptions about plate tectonics, rock formation, and other Earth-science mechanisms (Libarkin & Anderson, 2005).

That's a genuinely validated instrument, but it solves a different problem. It targets college-level geoscience content, not the map-reading and human-geography content a K-9 classroom actually teaches. Borrowing its name to describe a K-9 tool would overstate what's really available.

The Background Knowledge Probe: The Realistic Substitute

What a teacher can honestly borrow from these validated instruments isn't the instrument itself — it's the underlying principle. Anchor every pre-unit prompt to a misconception documented by real research, rather than guessing at what might trip a class up.

Classroom-assessment researchers Thomas Angelo and Patricia Cross documented exactly the tool for that job: the Background Knowledge Probe, a short, open-ended, ungraded prompt given before a unit to surface what students already believe about an upcoming topic (Angelo & Cross, 1993). It needs no answer key, no psychometric validation, and no map — just a well-aimed open question tied to a real, named belief.

Three Documented Geography Beliefs Worth Surfacing Before a Unit Opens

A pre-unit geography diagnostic doesn't need to invent misconceptions from scratch — cartography and geography-education research has already documented several, and an open prompt can surface each one before instruction starts. Reusing a documented belief here is honest, not lazy: the same misconception gets probed differently depending on why you're asking.

Does a Map's Shape Tell the Truth About a Country's Size?

Cartographers Lieselot Lapon, Kristien Ooms, and Philippe De Maeyer ran a worldwide study of more than 130,000 participants and found a measurable Mercator effect: people systematically overestimate the size of regions near the poles and underestimate regions near the equator, a bias traceable to lifelong exposure to Mercator-style web maps (Lapon, Ooms, & De Maeyer, 2020). This isn't a guess about what might confuse a class — it's a documented, quantified distortion in how people actually picture the world.

A pre-unit probe doesn't need a full projection lesson, just an open question that lets the belief show up unprompted:

"When you look at a world map, do you think the sizes you're seeing are always accurate? Pick two countries or continents you think you know well and guess how their real sizes compare."

A student who assumes the map's proportions are simply reality has revealed the exact bias Lapon, Ooms, and De Maeyer measured. That gives a teacher a concrete, real comparison to build the unit's first lesson around, without needing to correct a belief nobody knew was there.

Is Africa a Country?

Geography-education researchers Simon Catling and Steve Rawlinson, writing for the UK Department for Education's geography expert subject advisory group, named a specific, recurring error: casual conflation of continents, regions, and countries, the kind of muddling that treats "Africa" as a single uniform place rather than 54 countries (Catling & Rawlinson, 2014). It's a belief picked up as much from informal media exposure as from any classroom.

A direct, ungraded prompt surfaces it cleanly:

"Is Africa a country, or something else? Explain what you think it is in one or two sentences."

A response describing "a really big country" signals the exact conflation Catling and Rawlinson documented. A response correctly naming it a continent made of many countries matters just as much for planning where the unit's opening minutes should go.

What Does "Everyday Experience" Get Wrong About Physical Geography?

Geography-education researcher Jane Dove catalogued a wide set of physical-geography misconceptions rooted not in poor teaching but in plausible everyday experience — beliefs a student built from lived observation that happen to be wrong once a real mechanism is involved (Dove, 1999). A river running through hilly terrain near a home, for instance, can quietly teach a student the opposite of how river velocity actually behaves at different points along a course.

"Rivers usually move fastest right where they start, high up in the mountains, and slow down as they reach the flatter land near the sea. Do you agree or disagree, and why?"

A student who agrees has reasoned from a plausible, lived intuition — steep ground looks fast — rather than from the actual mechanics of channel shape and friction that make many rivers fastest in their middle or lower course. That's a genuinely different diagnostic signal than a simple factual gap, and it tells a teacher the first lesson needs to address an intuition, not just deliver a missing fact.

Documented BeliefNamed SourcePre-Unit Prompt Shape
A map's proportions are an accurate picture of real relative sizeLapon, Ooms, & De Maeyer (2020)"Do you think a world map's sizes are always accurate?"
Africa (or another continent) is a single countryCatling & Rawlinson (2014)"Is Africa a country, or something else?"
Physical processes work the way everyday, lived experience suggests, even when the mechanism says otherwiseDove (1999)"Rivers move fastest near their mountain source — agree or disagree?"

A Grade-Calibrated Illustration

Continue the Grade 6 world-regions scenario from the opening. Before a single lesson, you could hand out the three open prompts above on one half-sheet, unscored, with instructions that "I'm not sure" counts as a complete answer.

Reading twenty-eight short responses that evening beats building a full pre-test with a scored answer key. It also shows which belief is most widespread in this specific class, not just which one a textbook assumes matters most, and flags students who already reason correctly so the opening days don't reteach what they understand.

A Grade 2 class opening a simpler unit on "our world" can run a lighter version: "Do you think every place on Earth looks the same, or different? Why?" Younger students won't name projections or continents, but their answers reveal whether the class already senses real variation across places.

A Step-by-Step AI Workflow for Building a Geography Diagnostic

Building an honest pre-unit geography diagnostic starts with naming the specific belief you're probing, not the general unit topic, since a vague prompt collects vague, ungradeable answers.

  1. Name the specific belief, not the unit topic. "World regions" isn't specific enough; "believes a map's sizes are always accurate" is.
  2. Check whether the belief is already documented, in research like Lapon, Ooms, & De Maeyer (2020), Catling & Rawlinson (2014), or Dove (1999), rather than guessing at what might trip students up.
  3. Write the prompt as an open self-report question, not a multiple-choice item — "what do you think," not "which of these is correct."
  4. Ask AI to draft three or four open prompts at once, one per targeted belief, so the diagnostic samples broadly instead of narrowing to one misconception.
  5. Set your class profile's grade level and current region or unit, so the wording matches what students can express, even when the belief is more sophisticated than their vocabulary.
  6. Anchor domain coverage to your state's standard or to Geography for Life's Physical Systems and Human Systems Essential Elements (National Council for Geographic Education, 2012), so the diagnostic samples the unit's real scope, not just its headline term.
  7. Sort responses into three buckets before planning instruction: reasons correctly already, names the documented belief, or unsure — and let that sort, not a score, shape the first week's lesson plan.

A Sample Prompt

Grade 6 geography, opening a unit on world regions and human settlement.
Generate three short, open-ended, ungraded pre-unit prompts, one per
documented belief:
(1) a world map's visual sizes are always an accurate picture of real size
(2) a named continent (e.g. Africa) is a single country, not many
(3) a physical process (e.g. river speed) works the way everyday
    experience suggests, even when the real mechanism differs.
One or two sentences per prompt, answerable in under three minutes,
explicitly framed as ungraded.

You could use EduGenius to hold a class profile's grade level while drafting a set of open pre-unit prompts anchored to named geography misconceptions, then export the finished probe as a one-page handout or a slide to project on day one.

Tools for Building Geography Diagnostics

A tool for this job needs open-ended prompt generation and misconception-anchored drafting, not a scored answer key — a diagnostic's value comes from the belief a student reveals, not a grade. It also needs to stay out of the trap the geography quiz guide already flagged: never trusting AI to originate a real map image.

ToolOpen-Ended Prompt SupportAnchors to Named MisconceptionsClass-Context MemoryStarting Price
EduGeniusYes, via open-response format selectionYes, if the misconception is named in the promptYes, via saved class profiles25 welcome credits free; Starter $7.99/month (500 credits); Professional $15.99/month (1,000 credits)
General-purpose AI chat toolsYes, if explicitly requestedOnly if the source is supplied in the promptNo, unless re-typed each timeVaries by provider
Geoscience Concept Inventory (adjacent field)No — fixed instrumentYes, at full psychometric rigor, for college geoscienceNoFree to administer; not built for K-9 content
Paper Background Knowledge ProbeYes, inherentlyOnly as well as the teacher designs itNoNo cost

No tool replaces reading the raw responses by hand. The general diagnostic-speed guide covers a separate bottleneck, verifying what a prior teacher actually taught, in any subject. The geography true/false guide covers a related but distinct job: checking a graded claim after a belief has already been addressed, not surfacing it beforehand.

Pro Tips for Geography Diagnostics

  • Ask AI to name which documented belief each prompt targets. An untraceable prompt is a guess dressed up as a diagnostic.
  • Keep every prompt genuinely ungraded, and say so out loud. A student who suspects a "what do you think" question is secretly scored answers what they think you want, not what they believe.
  • Sample across map literacy, physical systems, and human systems in one sitting. You don't yet know which belief is widest in this class.
  • Reuse the same probe at the start of next year, and compare which beliefs recur. A misconception that shows up every year earns a dedicated first lesson.
  • Pair the map-size probe with a real equal-area comparison, like Greenland against Africa at true relative scale, so day one directly answers the belief the diagnostic surfaced (Lapon, Ooms, & De Maeyer, 2020).

What to Avoid

Presenting an AI-Drafted Probe as a Validated Instrument

An AI-drafted open prompt anchored to a real, named misconception is a reasonable classroom tool. It is not the Conceptual Inventory of Natural Selection or the Geoscience Concept Inventory, and calling it psychometrically equivalent overstates what a quick, honest check can claim (Libarkin & Anderson, 2005).

Scoring a Diagnostic Like a Quiz

Grading a "what do you think" response defeats its purpose and teaches students to hide their real thinking next time. Read it as a starting point for instruction, never a grade in the book.

Writing a Vague Prompt With No Named Belief Behind It

"What do you know about world regions?" collects scattered, hard-to-sort answers. "Do you think a map's sizes are always accurate?" targets one documented belief (Lapon, Ooms, & De Maeyer, 2020) and sorts cleanly into correct reasoning versus the Mercator-effect bias.

Letting AI Generate a New Map Just to Support the Diagnostic

The same rule the geography quiz guide already established for scored items applies here: a generated map image carries no guarantee its borders or proportions are accurate. A diagnostic prompt should describe a comparison in words, or point to a verified, sourced map, never a freshly generated one.

Key Takeaways

  • A diagnostic happens before instruction; a formative check happens during it. The formative guide checks whether today's teaching is landing; a diagnostic checks what a student already believes before any teaching starts this year.
  • Geography does not have its own validated diagnostic instrument, the way biology has the Conceptual Inventory of Natural Selection (Anderson, Fisher, & Norman, 2002). The nearest real precedent, the Geoscience Concept Inventory, targets a different, college-level field (Libarkin & Anderson, 2005).
  • The Background Knowledge Probe is the realistic model for an AI-assisted pre-unit geography diagnostic: lightweight, open-ended, ungraded, built for exactly this moment (Angelo & Cross, 1993).
  • Three documented beliefs are worth surfacing before a geography unit opens: map-projection size distortion (Lapon, Ooms, & De Maeyer, 2020), continent-country conflation (Catling & Rawlinson, 2014), and everyday-experience physical-geography beliefs (Dove, 1999).
  • Conceptual-change research explains why timing matters, not just scheduling: a misconception must be surfaced before instruction can genuinely displace it (Posner, Strike, Hewson, & Gertzog, 1982).
  • A diagnostic should sample broadly, not narrow to one belief, drawing from map literacy, physical systems, and human systems in a single sitting.
  • AI speeds up drafting misconception-anchored prompts; a human still reads the responses and decides which belief is widespread enough to shape the first week of instruction.

Frequently Asked Questions

Does geography have a validated diagnostic test like biology's CINS or physics's FCI?

No. Unlike biology's Conceptual Inventory of Natural Selection (Anderson, Fisher, & Norman, 2002) or physics's Force Concept Inventory, no widely validated instrument has been built specifically for K-9 geography's mix of map skills, physical systems, and human systems. The closest real precedent, the Geoscience Concept Inventory, targets college-level Earth science instead (Libarkin & Anderson, 2005).

What's the difference between a diagnostic assessment and a formative assessment in geography?

A diagnostic happens before a unit starts and surfaces what a student already believes, like assuming a world map's sizes are automatically accurate. A formative check happens during instruction and checks whether that day's teaching is landing in real time, largely gated by whether a map is already displayed.

Should a geography diagnostic be multiple-choice or open-ended?

Open-ended usually works better for a classroom-level pre-unit diagnostic. Building a validated multiple-choice instrument takes years of research the way the Geoscience Concept Inventory required (Libarkin & Anderson, 2005); a short, open self-report prompt surfaces the same kind of belief without that investment (Angelo & Cross, 1993).

How is this different from the general diagnostic-assessment speed guide on this site?

That guide solves a different bottleneck: verifying what a prior teacher actually taught. This guide covers which geography beliefs are worth surfacing before instruction, and why geography's honest answer differs from biology's validated-instrument precedent.

Sources

  • Anderson, D. L., Fisher, K. M., & Norman, G. J. (2002). Development and evaluation of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39(10), 952-978.
  • Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers (2nd ed.). Jossey-Bass.
  • Catling, S., & Rawlinson, S. (2014). Places to Tread Thoughtfully in Teaching Geography. Geography Expert Subject Advisory Group, UK Department for Education.
  • Dove, J. (1999). Theory into Practice: Immaculate Misconceptions. Sheffield: Geographical Association.
  • Lapon, L., Ooms, K., & De Maeyer, P. (2020). The influence of map projections on people's global-scale cognitive map: A worldwide study. ISPRS International Journal of Geo-Information, 9(4), 196.
  • Libarkin, J. C., & Anderson, S. W. (2005). Assessment of learning in entry-level geoscience courses: Results from the Geoscience Concept Inventory. Journal of Geoscience Education, 53(4), 394-401.
  • National Council for Geographic Education. (2012). Geography for Life: National Geography Standards (2nd ed.).
  • Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66(2), 211-227.
  • U.S. Government Accountability Office. (2016). K-12 Education: Most Eighth Grade Students Are Not Proficient in Geography (GAO-16-7).
#teachers#assessment#quiz