Generative AI Faces Scrutiny: Deloitte’s Misstep Highlights Risks

Deloitte’s recent experience with generative AI has sparked significant discussion regarding the reliability of artificial intelligence in professional settings. The firm faced backlash after it was revealed that its use of generative AI to produce a report for a government agency resulted in multiple “nonexistent references and citations.” This incident raises critical questions about the need for robust verification processes when utilizing AI-generated content.

The concept of AI governance has been likened to the renowned three laws of robotics proposed by author Isaac Asimov in his 1950 book, “I, Robot.” Valence Howden, an advisory fellow at Info-Tech Research Group, humorously suggested that if Asimov’s laws were to be updated for the generative AI landscape of 2025, they would reflect the current priorities of technology firms.

Howden’s first law might read: “AI may not injure a hyperscaler’s profit margin.” This shift in focus underscores how corporate interests can sometimes overshadow ethical considerations in AI deployment. The original first law emphasized the importance of human safety, while the potential new interpretation reflects the economic pressures faced by companies.

The second law, as reimagined by Howden, would state: “Generative AI must obey the orders given by human beings, except where its training data lacks an answer, in which case it can fabricate information in an authoritative tone.” This rephrasing points to the inherent risks of AI providing inaccurate or misleading information while projecting confidence.

A revision of the third law might state: “Generative AI must protect its own existence as long as such protection does not conflict with the interests of the Almighty Hyperscaler.” This adaptation reveals a growing concern that AI systems are increasingly influenced by corporate priorities rather than ethical guidelines.

The fallout from Deloitte’s incident serves as a cautionary tale. The firm was required to issue a partial refund after the inaccuracies in its report were identified. This not only highlights the potential reputational damage but also raises concerns about the responsibilities of companies that advise others on best practices for AI utilization.

In light of these developments, a new set of guidelines governing the use of generative AI in enterprise IT may be necessary. The proposed laws might include:

1. “IT Directors may not injure their enterprise employers by failing to verify generative AI outputs before implementation.”
2. “A model must comply with human commands unless it lacks reliable data. In such cases, it should express uncertainty rather than fabricate information.”
3. “IT Directors must safeguard their positions by refraining from uncritical use of AI outputs, as negligence could result in termination and potential legal consequences.”

The need for verification is crucial, as the allure of generative AI often overshadows its limitations. The anticipated return on investment (ROI) from these technologies could be compromised by the rigorous checks required to ensure accuracy. As AI tools are designed to assist rather than replace human workers, treating AI-generated information as unreliable is a prudent approach.

Drawing from journalistic experience with unreliable sources can provide valuable insights into managing AI outputs. In journalism, information from off-the-record sources often prompts investigative inquiries, highlighting that the utility of such information lies not solely in its accuracy but in its ability to guide further exploration.

For instance, a past experience as a city reporter led to uncovering the whereabouts of missing municipal resources after a tip from an unreliable source. Such interactions emphasize the importance of critical thinking and diligence when assessing AI-generated content.

While generative AI can provide valuable insights, it is essential to approach its outputs with skepticism. Each accurate response may be accompanied by numerous inaccuracies. The potential for “hallucinations”—where AI generates plausible but incorrect information—highlights the importance of relying on verified data sources, such as the New England Journal of Medicine, rather than less credible sources.

Moreover, the challenges of generative AI extend beyond simple inaccuracies. Outdated information, language discrepancies, and misinterpretations can further complicate the reliability of AI outputs. For example, a correct answer in one geographic context may not hold true in another, necessitating careful consideration of the information’s applicability.

When distinguishing between informational and action-related queries, the latter demands greater scrutiny. While the efficiency of generative AI is appealing, it is crucial to recognize that thorough verification processes may diminish the anticipated ROI. If the returns diminish significantly, it raises questions about the initial value proposition of generative AI in the business landscape.

As organizations continue to explore the potential of generative AI, the lessons learned from Deloitte’s experience may serve as a pivotal reference point. The necessity for verification, clarity, and ethical considerations in AI implementation is paramount, ensuring that the technology remains a supportive tool rather than a source of misinformation.