As AI use becomes more common, one of the concerns you may have heard about is AI ‘hallucinations’. Despite the name, an AI hallucination simply means that an AI system has generated information that is incorrect, misleading or unsupported, while presenting it as though it were accurate.
When AI is being used to support work involving a legal case, understanding why this can happen and how information can be checked is particularly important.
Why does AI sometimes get things wrong?
Generative AI works differently from a traditional search engine. Instead of just retrieving a stored answer, a large language model generates a response based on patterns it has learned from large amounts of data. It predicts what information is likely to come next based on the question it has been given and the context available to it. This means an answer can sound convincing even when it isn’t correct.
For example, if an AI system doesn’t have enough information to answer a question, it may still generate a plausible response instead of saying that it doesn’t know.
What can cause an AI hallucination?
Hallucinations can happen when an AI system is working with incomplete or unclear information, when a question is ambiguous, or when the model doesn’t have the information it needs to give an accurate answer. The way an AI system has been designed and the information it has access to can also affect the quality of its responses.
This is particularly relevant when working with medical records, where information can be complex, repetitive or spread across thousands of pages and multiple healthcare providers.
Why does this matter in the legal industry?
Accuracy matters in any use of AI, but the consequences of inaccurate information can be huge when working with legal and medical information. A date, diagnosis, treatment or clinical event could influence how someone understands a case. If you can’t check where an AI answer came from, you can’t be sure it’s accurate. This is why it’s important to verify information generated by AI rather than relying on the answer alone.
AI should help teams find and work with information more efficiently. It shouldn’t remove their ability to check that information for themselves.
Can AI hallucinations be prevented?
There isn’t a single solution that can guarantee an AI system will never produce an incorrect answer. However, AI systems can be designed in ways that reduce the risk and make their outputs easier to verify. One way to reduce the risk is to make sure the AI bases its answers on a specific source of information. When working with case information, this could mean using the documents within that particular case as the source for an answer.
Providing citations is another important safeguard. Instead of just presenting an answer, the system can show the user where the supporting information came from so they can review the original source themselves.
How does Ask Alex approach this?
Ask Alex, our AI assistant within MedBrief Insights, is designed to help users find information within their case documents. A user can ask Alex a question about their case and receive an answer based on the information available within those documents.
The response includes citations showing where the information has come from. Users can follow the citation back to the original record and check the information for themselves. This doesn’t mean professional judgement is handed over to AI. Instead, Ask Alex helps with the process of finding relevant information while keeping the user in control of how that information is interpreted and used.
AI has the potential to make working with large amounts of information significantly quicker. But speed isn’t useful if you can’t have confidence in the information you’re working with. Understanding that AI can get things wrong is part of using it responsibly.
For legal teams, it’s not just about whether a system uses AI, but how it uses their information, whether answers can be traced back to the source, and whether the user stays in control.
We believe AI should support professional judgement, not replace it. That’s why verification and transparency are an important part of how we’re building AI into the MedBrief platform.