Why Does ChatGPT Make Things Up About Brands?
ChatGPT makes things up about brands because it predicts the most likely next words instead of looking up facts, and its training rewards a confident guess over “I don’t know.” On OpenAI’s own test, a model that almost never declined to answer was wrong 75% of the time. Thin info about your store makes it worse.
Key Points
| Cause | What you see | Fix |
|---|---|---|
| The model predicts likely words | Facts borrowed from brands that look like yours | State your own facts in text on your site |
| Tests reward guessing | A confident answer when it should say “not sure” | None on your side, so plan around it |
| Thin or buried brand info | A wrong warranty, shipping, or return answer | Put policies in HTML, not PDFs or popups |
| Facts that disagree across the web | An old price or a discontinued product | Match your facts on every channel |
What Does a Made-Up Answer About a Brand Look Like?
A made-up answer about a brand looks like a confident product claim that was never true: a lifetime warranty you do not offer, a soy wax blend you never used, a store in a city you have never shipped from. OpenAI’s own definition of a hallucination is a “plausible but false” statement, and plausible is the dangerous half.
Under the hood, ChatGPT works like a very advanced autocomplete. Ask it about a small candle brand and it may say “soy wax” only because most candle sites that look like yours say soy wax. That answer is statistically likely, which is a different thing than accurate.
That’s also why the errors are so hard to spot. If ChatGPT cannot find your facts, it borrows facts from brands that resemble you. The claims are true for somebody, just not for you. The same resemblance logic drives which brands ChatGPT recommends in the first place.
ChatGPT Guesses Since Guessing Scores Points
ChatGPT guesses instead of saying “I don’t know” since the tests used to rank AI models score a blank answer as zero and a lucky guess as a win. OpenAI’s researchers made this case in Why Language Models Hallucinate, and they published their own models’ numbers to prove it.
| OpenAI model | Declined to answer | Answered wrong |
|---|---|---|
| o4-mini | 1% | 75% |
| gpt-5-thinking-mini | 52% | 26% |
Look at what that table rewards. The older model answered nearly everything and was wrong three times out of four, yet it scored slightly higher on accuracy, so it looked better on a leaderboard. The peer-reviewed version of the paper, published in Nature in April 2026, says the headline metrics “systematically reward guessing over admitting uncertainty.”
Outside testers see the same bluffing. Columbia’s Tow Center asked ChatGPT to identify the source of 200 news excerpts. It got 134 wrong, signaled doubt just 15 times, and never once declined to answer.
So you are not dealing with a bug that gets patched next month. A student who loses nothing for a wrong bubble fills in every bubble, and a model trained the same way does too.
Which Brand Facts Does ChatGPT Get Wrong Most Often?
The brand facts ChatGPT gets wrong most often are the ones no pattern can predict: prices, shipping cutoffs, return windows, warranty lengths, and stock. OpenAI’s paper explains that “arbitrary low-frequency facts, like a pet’s birthday, cannot be predicted from patterns alone,” and your policies are that kind of fact.
Spelling is the opposite case. The same paper notes that spelling and parentheses “follow consistent patterns, so errors there disappear with scale.” A bigger model learns grammar from billions of examples, but it sees your 45-day return window once, if it sees it at all.
So sort your facts into two piles before you fix anything. General claims about your category, like how soy wax burns or how merino handles sweat, tend to come out right since the whole web repeats them. The numbers only you can confirm are the ones to check first, and they are also the ones a shopper acts on.
Just know that no model will ever be perfect here. OpenAI says plainly that “accuracy will never reach 100%” since some questions cannot be answered from what the model has, which is one more reason to hand it the answer yourself.
Does Live Search Stop ChatGPT From Inventing Facts?
No, live search does not stop ChatGPT from inventing facts about your store. It only gives the model better material when better material exists. If your return policy sits in a PDF, a popup, or an image, the search comes back thin and the guessing takes over again.
To be fair, the trend is moving the right way. NewsGuard’s August 2025 audit found the ten leading chatbots repeated false claims 35% of the time, after their rate of declining to answer fell from 31% to 0% in a year. By its May 2026 audit, the average was down to 15.45% for typical prompts, and ChatGPT gave false information on 1 prompt in 20.
But those audits test big news stories that thousands of pages cover. Your shipping cutoff and warranty length live on one or two pages, and nobody else on the web repeats them. Small brands with thin footprints are where made-up answers still bite hardest, which is backwards from who can afford the damage.
The retrieval step is also pickier than most owners expect. I broke down what gets pulled, and what gets skipped, in my post on how ChatGPT picks its sources.
Fix the Gaps ChatGPT Fills In
Fixing the gaps ChatGPT fills in means publishing the same clear facts on every page it can read, starting with your own site. OpenAI’s crawler documentation says sites that opt out of OAI-SearchBot “will not be shown in ChatGPT search answers,” so the first check is whether it can read you at all.
I run this order for clients:
- Confirm your robots.txt is not blocking the AI crawlers that feed ChatGPT search.
- Put shipping, returns, and warranty terms in HTML text on their own pages.
- Write an FAQ page that answers real customer questions in full sentences.
- Make your About page say what you sell, who it is for, and where you ship, in one paragraph.
- Match your facts across marketplaces, social profiles, and retail partners.
With that list done, resist the urge to publish more. Five pages that agree do more than fifty that almost agree. An old blog post with last year’s return window is a second source, and the model has no way to know which one you meant.
The same goes for product pages. If the material, the size, and the weight limit are only in a photo or a spec PDF, write them in a sentence. Auditing a full catalog for those gaps is slow work, and it is a core piece of my AI search services for Shopify stores.
Can You Correct ChatGPT When It Gets Your Brand Wrong?
You can correct ChatGPT about your brand only by changing the pages it reads, since OpenAI’s help center lists no way for a brand to edit an answer. The good news is that answers built on live search refresh when the web does. OpenAI’s crawler docs say a robots.txt change takes about 24 hours to register for search results.
Content changes take a good deal longer than that in my client work. Publish the correct fact where it is easy to find, and the wrong answer usually fades within weeks, not days.
So make the check a habit. Once a month, ask ChatGPT the questions your buyers ask, like “does [brand] ship to Canada” or “what is [brand]’s return window,” and log what comes back. I laid out the full routine in my steps to track brand mentions on ChatGPT.
One more thing if you run an AI chat widget on your own store: you own what it says. A Canadian tribunal ordered Air Canada to pay a customer $812 after its chatbot invented a refund policy, and the ruling said it “makes no difference whether the information comes from a static page or a chatbot.”
Still, some SEOs will tell you made-up answers make AI search a waste of effort. I think the opposite. A machine that fills gaps rewards the brands that leave no gaps.
Want ChatGPT to Get Your Store Right?
ChatGPT makes things up when your web footprint gives it nothing better to say, so the fix is clear, matching facts on every page it reads. If you want to see where you stand, book a call and I will run your brand through the prompts your buyers use and show you which answers are wrong today. If you want the ongoing work handled, that is what my AI search services are for.
Sources
- OpenAI: Why language models hallucinate
- Nature: Evaluating large language models for accuracy incentivizes hallucinations
- Columbia Journalism Review, Tow Center: We compared eight AI search engines
- NewsGuard: August 2025 AI False Claim Monitor
- NewsGuard: May 2026 quarterly AI audit
- OpenAI: Overview of OpenAI crawlers
- CBC News: Air Canada found liable for chatbot’s bad advice
