In late 2025, the Australian government paid Deloitte — one of the Big Four accounting firms — about AUD 440,000 (roughly RM 1.3 million) for an independent assurance review of one of its welfare-penalty systems. Routine engagement: read the plumbing, write a report, deliver findings.
The report was filed. Then someone actually read it.
What they found was a small disaster. Several of the cited academic references didn't exist. A quote attributed to a federal court judgment had been invented wholesale. By the time the dust settled, roughly twenty fabricated items had been pulled out of the document.
Deloitte eventually issued a corrected version, acknowledged that ChatGPT had been used in drafting parts of the document, and refunded a slice of the contract. The story closed quietly enough.
What it leaves behind is a question that is about to land on most of the professions in this country, sooner or later: when AI gets it wrong, who is actually on the hook?
And Deloitte isn't the first one
Step back two years. In 2023, an American lawyer used ChatGPT to dig up precedent for a brief. The model — eager to please, as ever — produced a beautiful stack of cases. Judge names, docket numbers, quoted passages, all neatly assembled. The lawyer filed it without checking.
Opposing counsel went to look up the cases. They couldn't find any of them, for the very good reason that none of them existed. The court fined the lawyer USD 5,000, and the incident has become a standard cautionary tale in American law-school AI teaching ever since.
A bad day is a bad day. What both stories raise, underneath the mess, is a sharper question. When AI gets it wrong and somebody gets hurt, who is supposed to wear it?
It sounds simple. It isn't. At least three positions are defensible.
The AI company's line: We sold a product. The licence terms told you in plain text that the model can hallucinate. Verifying the output before you ship it is your job. Our duty to warn ended at signature.
The intermediary professional's line: The AI is essentially a black box. We can't see inside it. We could not have predicted exactly how this particular output was going to fall over.
The client's line: I hired you precisely so you would verify the work. I am not paying you to copy-paste from a chatbot.
All three are reasonable on their own. The trouble is that the first two cannot both be true at the same time. If they were, the whole arrangement collapses into a vacuum — each party gets to point at the next, and the harm ends up owned by no one. Legal scholars have a name for this trap; they call it the Double Shield.
We've seen this movie before — with pills
Take the AI out of it, and this is just an old problem in a new outfit. The earlier version of the argument played out around prescription drugs.
Strip it down. You go to the doctor. She prescribes you a drug. You take it. Something goes badly wrong. Whose fault is it — the drug company's, or the doctor's?
For more than half a century, American courts have given a steady answer. As long as the manufacturer has been straight with the prescribing doctor about risks, side effects and contraindications, the bulk of its duty is done. The rest sits with the doctor — because the doctor is the one who actually knows your case, weighs the drug against your specific history, and decides whether to write the prescription in the first place.
The doctrine has a name. American courts formalised it in a 1960s pharmaceutical case and have called it the Learned Intermediary Doctrine ever since. The intuition is plain: for the system to function at all, there has to be a competent human standing between the manufacturer and the patient, doing the judging.
What gives the competent human any value at all is precisely that judging. The moment the doctor stops thinking and just slides the manufacturer's brochure across the desk, she has stopped being an intermediary in any useful sense. She has become a courier. And once she's a courier, the law can take her intermediary status away.
Now run the same logic on AI
When a firm hands an AI-drafted report to a government client without verifying it, or a lawyer files AI-generated case law without checking, they are not "users". They are intermediaries. The whole reason they were hired is to be the competent human between the AI and the client. Skip that step, and they have stopped being intermediaries. They have become couriers — extremely well-paid ones.
And the second the intermediary stops doing the judging, the shield the AI company had been hiding behind collapses with it. The entire architecture of immunity rested on the premise that a competent human was, in fact, doing the judging. If nobody is, the shield has nothing to stand on.
So the honest answer to "whose fault?" is rarely "the AI company", and rarely "the client got what they deserved". It is usually the professional in the middle — the one who was meant to be the gatekeeper, and walked off the gate.
This isn't only consultants and lawyers
The logic stretches everywhere a professional touches an AI output. A teacher running student work through AI. A doctor reading scans with one. A scientist drafting a paper with one. An engineer letting one write the code. The pattern is identical: if you are a professional, and you pass AI-generated output to people who are relying on you, you are the intermediary. The mistakes land in your lap.
The duty is proportional, not absolute. An internal working draft doesn't need to be combed through line by line. But the moment the output goes to a client, a patient, or a court — the moment somebody's money, health, or liberty is in play — verification stops being best practice. It becomes the floor.
I see this play out in my own field. AI is routine in scientific research now, but the model will occasionally produce a citation or data point that looks impeccable — until you actually try to chase it down, at which point it simply isn't there. The more a project leans on AI, the more careful the pre-submission checking has to be, not less. If anything ever slips through to publication, blaming the AI company is impossible. It would also be wrong.
Pulling this back to Malaysia is worth doing. Consulting, accounting, law, medicine, education — every one of these sectors here is already pulling AI into daily work. If our professional codes and industry self-regulation don't keep pace with the tools coming in, there is no good reason to think this kind of incident will stay an Australian story.
Can regulation take some of the load? Yes, some of it. The European Union's AI Act, passed in 2024, classifies AI in courts, classrooms and clinics as "high-risk", and subjects those deployments to upfront review and transparency requirements. It also explicitly demands that high-risk systems be designed so that humans can actually oversee them — meaning the human in the loop is supposed to oversee, not be furniture. All welcome. But regulation only ever sets a floor. It cannot stand in for the individual doctor, lawyer, or teacher making the specific judgment call on the specific AI output sitting in front of them.
So — will AI replace the professional?
This is the question quietly running through every profession right now.
The answer is not "replace". The answer is "reframe".
For most of modern history, the professional's value lay in being able to do what others could not. Today, AI can do a fair amount of what "others could not". But there is one thing AI cannot do, and the professional still must — bear responsibility. Legal, ethical, social responsibility has to land on a person. Only a person can be sued, lose a licence, stand in front of the mirror and answer for a judgment call.
So the meaning of "professional" has to shift in the AI era. It is no longer just "can do." It is "will be accountable for what was done."
AI can help you write. It cannot sign for you. And a signature, in the AI era, carries more weight than it ever has.
This essay draws on conversations with Blaise Tayese, a law student under the author's supervision, and on his recent working paper "The Duty to Verify". Gratefully acknowledged.