Digital Business After 40

AI without the overwhelm

How Can You Tell If AI Is Wrong? The 5-Minute Check I Run on Every Answer

AI gets things wrong and never sounds unsure. Here's the 5-minute way to fact-check any answer, and why your decades of judgment catch what it invents.

Tina Alexander, founder of Digital Business After 40 By Tina Alexander

AI will never tell you it’s guessing.

It doesn’t pause. It doesn’t say “um.” It hands you a made-up number in the same calm, confident voice it uses for the right ones. That’s the whole problem, and it’s why so many beginners get burned.

Here’s the short answer: you can tell if AI is wrong by checking the four things it gets wrong most (names, numbers, sources, and anything recent) with a quick 5-minute routine. Read it out loud. Look up the one fact that matters most. Ask where the information came from, then go see that source yourself. If you can’t confirm it, cut it.

No tech skill needed. What you need is judgment, and if you’re over 40, you’ve been building that for decades.

If you’re brand new to AI, start with the complete beginner’s guide to AI after 40. This article takes one piece of that guide, knowing when AI gets it wrong, and turns it into a habit.

Key takeaways

  • AI doesn’t lie on purpose. It guesses, and it never sounds unsure while it does.
  • It gets four things wrong most: names, numbers, sources and recent events.
  • The 5-minute check: read it out loud, spot-check one fact, and verify the source yourself.
  • Asking the AI “is that true?” isn’t proof. Check outside the chat.
  • If you can’t verify it, cut it. Don’t soften it.

Why does AI sound so sure when it’s wrong?

Because it was trained to answer, not to say “I don’t know.”

OpenAI, the company behind ChatGPT, said this plainly in a 2025 research paper. Their explanation is that the way AI gets tested rewards guessing. Think of a multiple-choice exam. Leave a question blank and you get zero. Guess and you might get lucky. So the AI learns to guess.

AI isn’t looking facts up the way you’d look something up in a book. It’s predicting what a good answer usually looks like. Most of the time, that prediction is close enough. When it isn’t, you get something that looks exactly like a real answer and just isn’t one.

That’s why I don’t call it lying. Lying means you know the truth and hide it. AI doesn’t know. It fills the gap with whatever sounds right.

And “sounds right” is exactly what fools people.

What does AI get wrong most often?

In my experience, and in the research, it comes down to four things.

1. Names. People, books, companies, court cases. AI will happily invent a book title and give it a believable author.

2. Numbers. Statistics, prices, percentages, dates. A made-up number looks just as solid as a real one. Often more solid, because made-up numbers tend to be neat and round.

3. Sources. This is the big one. In one study from the Canadian Psychological Association’s student journal, a researcher asked ChatGPT for scientific references. About a third of them didn’t exist. The titles looked real. The authors looked real. The papers weren’t there.

4. Anything recent. AI tools learn from information up to a certain date. Ask about this year’s prices, rules or news, and it may give you last year’s answer with this year’s confidence.

This isn’t a problem with one tool, either. In 2025, the BBC and the European Broadcasting Union tested ChatGPT, Copilot, Gemini and Perplexity on more than 3,000 news answers across 18 countries. 45% of the answers had at least one significant issue. Nearly a third had problems with their sources.

So when someone tells you “just use a better AI,” that’s not the fix. The fix is you.

What is the 5-minute check for AI answers?

This is the routine I run on anything AI writes for me before it goes out with my name on it. It takes about five minutes. Often less.

Step 1: Read it out loud. Your ear catches what your eyes skim past. When a sentence sounds polished but doesn’t actually say anything, you’ll hear it. When a claim sounds a little too tidy, you’ll hear that too.

Step 2: Spot-check the one fact that matters most. You don’t have to check every word. Pick the claim the whole piece leans on: the number, the name, the price, the rule. Search for it yourself. If that one holds up, you’re usually in good shape. If it doesn’t, the rest needs a closer look.

Step 3: Ask “where did that come from?” and then go look. Ask the AI for its source. Then open that source yourself, outside the chat. This second half matters. In 2023, a New York lawyer asked ChatGPT whether the court cases it gave him were real. It told him they did indeed exist. They didn’t, and he and his colleague were fined $5,000. The AI saying “yes, it’s real” is not the same as you seeing it’s real.

Here’s a prompt you can paste in after any answer:

“For every fact, number, name and quote in your answer, tell me where it came from. If you’re not sure, say ‘not sure’ instead of guessing.”

It won’t catch everything. It does make the guesses easier to spot.

If you want your answers to start out stronger, so there’s less to check, it helps to ask well in the first place. I walk through that in the four-part AI prompt that doesn’t give you mush, and you can grab AI Without the Overwhelm, my 12 free starter prompts to try it right away.

Why are people over 40 so good at catching AI mistakes?

Because you’ve heard a lot of confident answers in your life. And you’ve learned that confident and correct aren’t the same thing.

I spent years trying to make money online before anything worked. I paid a lot of money along the way. That taught me a lesson I didn’t want, and now I’m glad I have it: the smoother something sounds, the more closely I look.

That’s the exact skill fact-checking AI takes.

A 25-year-old who grew up with smartphones might be faster at the tech. But speed isn’t the skill here. Judgment is. You know what a sales pitch that’s too smooth sounds like. You know when a story doesn’t add up. You’ve been reading people for decades.

AI is just one more voice that sounds very sure of itself. You already know what to do with those.

Where does a wrong AI answer actually cost you?

Not every mistake matters. A slightly off word in a brainstorm? Who cares. But three places deserve the full 5-minute check every single time.

Prices and money. Fees, tax rules, what a platform charges, what something costs. Get these wrong and real people pay for it, including you.

Claims about results. Anything that says your product “will” do something, or quotes a statistic to make a point. If you can’t back it up, it doesn’t go on your sales page.

Anything with your name on it. Blog posts, emails, book descriptions, social posts. When you publish it, it’s yours, not the AI’s. Readers won’t blame the tool. They’ll remember you. If you publish books, this goes double, and I’ve covered how to disclose AI content on Amazon KDP so you stay on the right side of their rules.

What should you do when you can’t verify something?

Cut it. Don’t soften it.

The temptation is to keep the fact and add “around” or “studies suggest.” That doesn’t fix anything. A made-up number with “about” in front of it is still a made-up number.

If you can’t find where it came from in five minutes, take it out. Your piece will almost always be fine without it. Usually it’s better, because what’s left is what you actually know.

Questions people over 40 ask about fact-checking AI

Does AI make things up?

Yes. AI tools like ChatGPT and Claude predict what a good answer usually looks like, so when they don’t know something, they often fill the gap with something that sounds right. That can be a made-up number, a book that doesn’t exist, or a quote nobody said. It happens most with names, numbers, sources and recent events.

How do I fact-check an AI answer quickly?

Run three checks. Read the answer out loud and listen for anything that sounds smooth but says nothing. Pick the one fact that matters most and look it up yourself. Then ask the AI where it got the information, and open that source with your own eyes. If you can’t find it, cut it.

Can I just ask the AI if its answer is true?

Asking is a good start, but it isn’t proof. AI can repeat a wrong answer just as confidently the second time. In one well-known court case, a lawyer asked ChatGPT if the cases it gave him were real, and it said yes. They weren’t. Always check the source yourself, outside the chat.

Is ChatGPT more accurate than other AI tools?

No single AI tool is always right. A 2025 study led by the BBC tested ChatGPT, Copilot, Gemini and Perplexity and found significant problems in 45% of their news answers. Whichever tool you use, the same habit protects you: check the facts that matter before you publish or act on them.

Do I need tech skills to spot AI mistakes?

No. Spotting AI mistakes is a judgment skill, not a tech skill. If you’ve ever heard a sales pitch that sounded too good, or a story that didn’t quite add up, you already have the instinct. The 5-minute check just gives that instinct a routine to follow.

Your judgment is the part AI can’t fake

AI is fast. It’s helpful. I use it every day, in ChatGPT, Claude and Muse.

But it will never tell you it’s guessing. That part is your job.

And you’re good at it. You’ve spent decades telling the difference between sounding sure and being right. That doesn’t go out of date. It’s the one skill these tools can’t copy.

So the next time AI hands you a neat number with a source attached, read it out loud. Check the one fact that matters. Go see the source with your own eyes.

AI brings the speed. You bring the judgment. Your age isn’t a disadvantage here. It’s the advantage.

If you want people around you who are learning this too, come join us in the free Digital Business After 40 community. Bring the most confidently wrong thing AI has ever told you. We’ll check it together.