Cautionary Tales of Using AI Chatbots: The Courts Weigh in on AI Hallucinations

Legal practitioners are increasingly relying on AI (Artificial Intelligence) language models for assistance. Fasken’s recent bulletin, "Cautionary Tales of Using AI Chatbots: The Courts Weigh in on AI Hallucinations," suggests this rising trend. Yet, at its core, the bulletin conveys a crucial insight: AI is not intelligent. 

The Illusion of Artificial Intelligence

When we think of AI, we envision a sophisticated entity capable of reasoning, learning, and understanding complex legal concepts. However, the current reality is different. AI, including language models like Chat GPT, operate by predicting the most probable next word or piece of information based on patterns. It excels at recognizing patterns and making probabilistic guesses. AI lacks true comprehension. 

The Case of Zhang v Chen

The bulletin cites Zhang v Chen as a stark reminder to tread carefully when relying on AI-generated content.

In this case, a lawyer’s overreliance on AI led to a legal submission citing two entirely fabricated cases hallucinated by ChatGPT. Justice Masuhara’s decision underscored the lawyer’s naivety in blindly trusting AI. The lawyer's insertion of these fictitious cases constituted an abuse of process, akin to making false statements to the court, which could result in a miscarriage of justice, eroding the integrity of the legal process. While AI excels at recognizing patterns, it lacks genuine understanding and should be treated as a tool rather than an infallible advisor. Instead, it requires supervision, much like the metaphor of “an eager but inexperienced intern or associate”. Importantly, Proper supervision and critical assessment would have prevented an error like this from occurring.

The bulletin highlights the risks associated with unsupervised AI usage, which can indeed lead to unforeseen consequences. However, it is essential to recognize that when AI is wielded judiciously and with proper understanding, it transforms into a valuable tool —AI, though brilliant at guessing, remains a creation of our making. Its brilliance lies in our discernment.

Cautionary Tales of Using AI Chatbots: The Courts Weigh in on AI Hallucinations Tracey M. Cohen, K.C., Alexandra Mitretodis, Raylene Smith. Fasken Bulletin, 03.12.2024.