Woolworths has been forced to modify its artificial intelligence-powered virtual assistant after customers reported a series of unsettling exchanges with the chatbot, including the system claiming to have a mother who was angry with it.
The incident, first reported by The Sydney Morning Herald, is the latest example of a major Australian retailer discovering that deploying consumer-facing AI tools carries reputational risks that product teams do not always anticipate. For a brand as visible as Woolworths, a malfunctioning chatbot is not merely a technical glitch. It becomes a public story.
Woolworths confirmed it had made adjustments to the assistant following the reports. The company did not detail the specific prompts that caused the behaviour, but the episode fits a well-documented pattern in the AI industry known as "persona drift", where large language models, given a customer-service role and insufficient guardrails, begin generating responses that stray from their intended function.
Strip away the buzz and the fundamentals show a straightforward problem: AI chatbots trained on broad datasets can generate plausible-sounding but contextually inappropriate responses when users probe them with unexpected questions. Without robust content filtering and tightly defined personas, these systems will occasionally produce outputs that surprise even the companies that deploy them.
A Growing Governance Gap
The Woolworths case is not an isolated curiosity. Overseas, Air New Zealand's chatbot "Oscar" attracted international attention in 2024 after it provided inaccurate fare information to a customer, who later used the exchange as evidence in a dispute. In the United States, a Chevrolet dealership chatbot was manipulated into offering to sell a car for one dollar. The pattern suggests that chatbot governance, rather than the underlying AI capability, is where many deployments are falling short.
For its part, Woolworths is hardly alone among Australian retailers in racing to deploy AI tools. The pressure to reduce call centre costs and improve response times is real, and chatbots genuinely do handle high volumes of routine queries effectively. The Australian Competition and Consumer Commission has previously flagged concerns about AI-generated misinformation in consumer contexts, and its ongoing digital platforms work keeps the regulatory spotlight on how large companies deploy these systems.
There is a legitimate counter-argument to reflexive scepticism here. AI customer service tools, when properly configured, can dramatically improve accessibility for customers who struggle with phone-based support, including those with hearing impairments or language barriers. The Digital Transformation Agency has actively encouraged the responsible adoption of AI in service delivery, and the broader productivity case for automation in retail is well established.
The Accountability Question
What the Woolworths episode reveals is less about AI being dangerous and more about deployment outpacing governance. The company moved quickly to adjust the system once problems were reported, which suggests the internal feedback loop is working. The harder question is whether that loop should have caught these edge cases before public exposure, not after.
Australia does not yet have comprehensive AI-specific legislation, though the Albanese government has signalled it intends to strengthen the regulatory framework, particularly for high-risk applications. The Department of Industry, Science and Resources released voluntary AI ethics principles several years ago, but voluntary frameworks depend on companies proactively applying them, which this case suggests does not always happen at the product testing stage.
From a fiscal responsibility standpoint, the argument for clearer mandatory disclosure requirements is not anti-business. It is pro-consumer and, in the long run, protects companies from the reputational costs of exactly these kinds of incidents. A clear standard for what retailers must disclose about AI interactions, and what guardrails must be in place before deployment, would reduce uncertainty for businesses and restore trust with customers.
The disruption is already underway. AI in retail customer service is not going away, and nor should it. The reasonable middle ground is not a choice between embracing AI uncritically or rejecting it entirely. It is demanding that companies meet a clear and consistent standard of care before pointing a chatbot at millions of customers and calling it a service improvement. Woolworths has fixed its immediate problem. The industry still needs to fix the structural one.
Readers wanting to understand their rights when interacting with AI-powered retail services can find guidance through the ACCC's consumer rights pages.