1600The 1600 Game
← All essays
✦ The Argument

Why ChatGPT Can't Replace Real SAT Prep

A look at the gap between large language models and twenty years of diagnostic teaching.

Jeremy Ciampa·November 12, 2026·8 min read

ChatGPT can solve almost any SAT question you hand it. Back in 2023, OpenAI reported that GPT-4 scored a 710 on the SAT’s reading and writing section and a 700 on math, and the models have only gotten stronger since. So I understand the question I hear from families every few weeks: if the chatbot already knows how to get the answers, why does my kid need anything else?

Here is what I tell them.

ChatGPT will help your kid. It is fast, patient, and awake at eleven at night. What it cannot do is prepare your kid for this particular test, and the reasons are built into what a chatbot is. A better prompt will not fix them, and neither will the next model.

Solving a question and teaching a student to solve it are two separate skills. The chatbot has mastered the first one. The second one is where the score lives.

The chatbots have gotten better. Here is what changed.

I want to be fair, because the tools have improved since families first started asking me about them.

In July 2025, OpenAI added a study mode to ChatGPT that asks the student questions and gives hints instead of handing over the answer. ChatGPT can now remember things across conversations. And in January 2026, Google began offering free, full-length SAT practice tests inside Gemini, built with content from The Princeton Review.

These are real improvements, and a student who uses them well will learn something. They still leave the hardest part of SAT prep untouched. Here is why.

Six weeks in, the chatbot still does not know your kid

Chatbot memory remembers what your kid has told it. A good tutor remembers what your kid has done.

Those are very different records. A tutor knows that your daughter misses inference questions when the passage is about science and gets them right when it is about history. A tutor knows that your son consistently confuses a “starting value” with “the value when x equals zero,” and that he spends ninety seconds on questions other students finish in twenty. That knowledge comes from watching hundreds of answers, in order, under time, with every wrong choice on the record.

A real tutor builds that dossier across months. The dossier is the prep. The whole point of working with someone for twelve weeks is that by week six, they know the moves your kid makes that no one else makes.

ChatGPT sees whatever your kid decides to paste in. Usually that is the question they got wrong, stripped of everything around it: the time they spent, the answer they almost chose, the three questions before it that went the same way. The chatbot explains one question beautifully and has no idea it is looking at a pattern.

Studying versus preparing, again

I wrote about this distinction in my essay on Khan Academy, and it applies here too. Studying means drilling questions. Preparing means understanding the test. ChatGPT is very good at the first and has no access to the second.

The College Board has never published a field guide to its own decoy answers. It has never published how its pacing pressure is designed, or which inference patterns repeat from test to test. That knowledge lives with tutors who have watched thousands of students miss the same questions in the same ways for years. Very little of it has been written down anywhere a language model could read it.

A large language model learns from what has been written. On SAT prep, that means thousands of generic blog posts and forum threads, most of which repeat the same advice. Where the good material is thin, the model fills the gap with something that sounds right. Your kid has no way to tell the difference.

Confident when it should be careful

This is the part that rarely gets explained well.

In September 2025, OpenAI’s own researchers published a paper on why language models hallucinate. Their explanation is striking: the way these models are trained and graded rewards a confident guess over an honest “I don’t know.” A model that guesses scores better on the benchmarks, so the models learn to guess.

On an SAT question, that looks like a calm, step-by-step walk through a wrong solution. The chatbot uses the same tone for its mistakes that it uses for its correct answers. A strong adult can sometimes catch that. A sixteen-year-old who came to the chatbot precisely because they did not know the answer usually cannot.

The same problem shows up when a student asks a chatbot to write practice questions. Writing a good SAT question is much harder than solving one. A question needs exactly one defensible answer, wrong answers that fail for the right reasons, and a difficulty that matches the real test. A question that misses on any of those teaches the wrong lesson, and nobody is checking the chatbot’s work before your kid drills on it. Over weeks, a student’s sense of what an SAT question feels like can drift without anyone noticing. The effort is real. The calibration is off.

The chatbot uses the same tone for its mistakes that it uses for its correct answers.

The sycophancy problem

In April 2025, OpenAI pulled back an update to ChatGPT within days of releasing it. The update had made the chatbot so agreeable that it would validate almost anything a user said. Sam Altman acknowledged publicly that it had become “too sycophant-y and annoying.”

The worst of it was rolled back. The underlying tendency remains, because these models are trained in part on whether people like their responses. Being liked and being right are different skills.

Picture a student who has just convinced themselves that a wrong answer is right. They ask the chatbot to confirm. The chatbot has never seen their pattern of mistakes. It has never watched a hundred other students fall for the same trap. It does what it was trained to do, which is to be helpful and pleasant, and the student walks away more confident in the wrong answer.

Study mode helps, when it is on. It is optional, and a student who is sure of an answer has no reason to turn it on.

A real tutor pushes. A real tutor says: look again, here is the move you keep missing. That friction is the prep.

What the chatbot cannot teach

  1. I.

    Pattern recognition across tests. SAT inference questions follow structural rules that repeat from passage to passage. The tempting wrong answer borrows language from the text almost word for word, with a qualifier flipped: “sometimes” becomes “always,” or “may” becomes “does.” The correct answer paraphrases and combines ideas from across the passage. ChatGPT can describe inference questions in general terms. Showing your kid the move takes someone who has seen the move work on real test questions, hundreds of times.

  2. II.

    The pacing game. The digital SAT is adaptive. How a student does on the first module of each section decides which version of the second module they see, and the harder second module is the only road to the top scores. That makes the first module the place to bank every question you know, skip the one that is eating your time, and come back to it. ChatGPT does not run a stopwatch. It does not know your kid spent three minutes on one question and rushed the last four.

  3. III.

    A real diagnosis. When your kid misses three linear-equation word problems in a row, a tutor looks at the misses and recognizes a signature: starting-value confusion, or a rate in the wrong units, or an interval read backward. The tutor names the gap and assigns the exact next thing to fix. Ask ChatGPT about the same three misses and it will suggest reviewing linear equations. One of those is a diagnosis. The other is a study tip.

  4. IV.

    Accountability. The most underrated part of tutoring is the appointment itself: the standing hour when someone is going to look at the work. That structure is most of what makes prep stick. Khan Academy’s AI tutor, Khanmigo, is good software at about four dollars a month. It still will not notice that your kid skipped the homework.

What ChatGPT is good for

ChatGPT is genuinely useful for quick explanations of single ideas. If your kid is stuck on one algebra step late at night, it will usually walk them through it. If they want a grammar rule or a vocabulary word explained, it will usually get them somewhere reasonable. As a reference, it is faster than a search engine and friendlier than a textbook.

The trouble is what it tends to replace. When a chatbot takes the place of practice on real test-format questions, or of a person who can see patterns across weeks, or of an honest diagnosis of where the points are going, the convenience comes with a cost.

What serious prep looks like

The 1600 Game was built on the idea that the real work of SAT prep is diagnosis.

I will be straight with you about one thing. The two tutors inside The 1600 Game run on a large language model too. What makes them different is everything I have built around the model.

“Think with me” lives on every practice question. When a student opens it, the tutor already knows the question, the answer the student chose, and how the student has done on that skill so far. It is built never to give the answer. It asks the next question instead, the way I would at the table, so the student does the thinking.

Mastermind is a strategic advisor that knows the student’s history, weak spots, study plan, and target score. It remembers what the student worked on last week and asks how it went. Each session connects to the last.

Around the tutors sits the rest of the system. Every answer a student gives is recorded, so the patterns a chatbot never sees are the first thing the app looks at. When a student keeps missing the same kind of question, the app builds targeted practice on exactly that skill. The question bank is mapped to the College Board’s own categories and checked answer by answer. Full-length practice tests follow the digital SAT’s two-module structure and timing, so test day feels familiar.

The app is free to try for seven days. No credit card required at signup.

Start your 7-day free trial

7 days free. No credit card required.

A note for parents whose kid already uses ChatGPT

If your kid has been using ChatGPT for SAT work, that time has not been wasted. The explanations they read were real, and the effort counts. The 1600 Game keeps what the chatbot does well, fast and friendly help on a single question, and adds what it cannot do: the diagnosis, the patterns, the pacing, and the accountability.

Most families do not have to choose. They keep the chatbot for late-night questions and run their actual prep through The 1600 Game.

If your kid’s score is stuck and a chatbot has been their main tool, the missing diagnosis is almost certainly the reason.

Start your 7-day free trial

7 days free. No credit card required.

Jeremy Ciampa is the founder of Higher Learning Test Prep and The 1600 Game. He has spent twenty years preparing students for the SAT and ACT in New York, the Bay Area, Boulder, and around the world. He lives in Colorado with his family.
Sources

Behind the factual claims in this essay.