When AI gives wrong answers, restaurant staff pay the price

Madison, a server in New York City, has noticed something troubling: customers are increasingly checking her answers against ChatGPT, an AI chatbot that can generate human-like text and answer questions. The problem? Sometimes the AI gets things wrong.

When Madison tells diners that a dish contains shellfish — a serious allergen that can trigger dangerous allergic reactions — some guests push back. Their reason: ChatGPT told them there's no shellfish in the dish. "They just want to talk to ChatGPT," Madison explains. The AI isn't actually checking a restaurant's recipe; it's making educated guesses based on patterns it learned during training. When it guesses incorrectly, it's called a "hallucination" — basically, the AI confidently stating something false.

This creates a dangerous gap between what an AI thinks might be true and what's actually on the plate. Restaurant staff have spent years learning about ingredients and food safety. But now, some customers are treating a machine's guess as more trustworthy than a person's expertise. The real risk isn't just rude customers — it's that someone could eat something they're allergic to because they believed an AI over a trained professional.

As AI tools become more common in everyday life, this story highlights a bigger problem: people often trust AI without realizing it can be confidently wrong. For service workers and anyone relying on AI for important decisions — from health questions to safety issues — that's a serious concern.