Navigating the Complexity of AI Documentation in Enterprise Decision-Making
Insights from the Watson Grinding explosion case involving 3M have sparked a re-evaluation of how organizations approach prompt governance in AI use. In this instance, an engineering expert engaged ChatGPT while formulating an analysis. Among the exchanged prompts, one notably asked the AI to “show how 3M is 0% at fault.” While it's still unclear if 3M explicitly instructed the expert to employ ChatGPT or to enter that exact prompt, the ramifications of the AI interaction are significant. Once the AI-generated material appeared in the case, it highlighted the potential complexities surrounding accountability and transparency in AI-assisted work.
During depositions related to this case, attorney Will Moye shared that he faced hurdles when probing the expert about ChatGPT-generated materials. After some back-and-forth, he received over 350 pages of ChatGPT interactions that had not been disclosed before. This shift from examining the expert's report to scrutinizing the underlying dialogue illustrates a crucial point: the nature of accountability is evolving, with AI interactions becoming integral to understanding decision-making processes.
The Role of Prompts in Decision-Making Records
Historically, enterprises have concentrated on the risks associated with what information they feed into AI systems. Employees have been cautioned against sharing sensitive data or proprietary content with generative AI. These precautions are essential, particularly in regulated industries. However, attention to inputs is only part of the governance equation; organizations must also consider how AI interactions represent a broader context for business decisions.
Take for example a scenario where an engineer uses an AI assistant to evaluate various design options. The iterative prompts given to the AI capture not just preferences but also assumptions and decision points that may never surface in the final output. The prompts serve as a digital breadcrumb trail, adding depth to the reasoning behind decisions, despite the risk of misinterpretation if viewed in isolation. Context becomes key in understanding the relationship between the prompt and the final decision, particularly when AI outputs influence critical outcomes.
The American Bar Association recognizes AI chat logs as potentially valuable discovery material. These logs can elucidate questions, hypotheses, and reasoning that inform final outputs, thereby enhancing operational governance discussions. However, the absence of clear standards around the retention and management of AI-generated content places organizations at a crossroads of responsibility and risk management.
Lifecycle Management: Beyond Input Protection
The revelations from the Watson case have shifted my inquiries concerning enterprise AI. Where I previously focused on the data input into models, I now find myself equally concerned about the records generated through usage. This forward-thinking question pushes governance discourse beyond mere acceptable-use policies and into the realm of comprehensive information lifecycle management.
Crucially, AI's involvement doesn't conclude once the model outputs an answer. Organizations must grapple with how conversations are documented, stored, shared, and ultimately deleted. For instance, OpenAI's shared links for ChatGPT indicate that conversations can be viewed beyond individual workspaces, raising significant governance issues when multiplied across an organization using multiple AI applications. Each AI tool comes with disparate retention policies, administrative controls, and sharing capabilities, complicating compliance and governance efforts.
It's clear that retaining every prompt is neither feasible nor wise. Organizations often drown in documentation without ample clarity on what truly matters. Effective governance comes from understanding the consequences of work output rather than attempting to capture every detail. Distinct scenarios require different levels of documentation: a simple email clarification shouldn't warrant the same scrutiny as an engineer's safety analysis driven by AI interaction.
Establishing Accountability Through Reconstruction
Adopting a backward-thinking approach—considering what evidence might be necessary for future evaluations—can strengthen accountability frameworks. Should a challenge arise months later, organizations must be able to reconstruct the decision process: what information informed the decision, how AI played a role, and who was responsible for the ultimate outcome.
The legal landscape surrounding AI interactions continues to evolve, and not every prompt will become discoverable. Factors such as privilege and relevance affect the discoverability of AI-related material, as highlighted by a New York court's recent ruling regarding ChatGPT records related to legal research. However, rather than ignore these complexities, organizations must anticipate governance challenges proactively.
While I don't conclude that every AI prompt should be regarded as a record, the 3M case does underscore the importance of recognizing when an AI interaction holds enough weight to warrant documentation. As Josh Copeland, a cybersecurity expert, succinctly puts it, “AI won’t testify for you … but it will absolutely testify against you.” Hence, preparedness is key: organizations must be ready to reconstruct their decisions under scrutiny.
The central question every organization employing AI should grapple with is: If this decision were contested a year later, could we effectively retrace how it was made?
