Putting ChatGPT on the Stand as an Expert Witness
How the use of AI in legal proceedings might be the herald of AGI
A judge asked ChatGPT if the term "landscaping" might cover the installation of a "ground-level trampoline"1. He was ruling on a lawsuit involving a landscaper and his insurance company. The insurance company wouldn't pay for damages incurred by the landscaper while installing said trampoline, because the policy covered only "landscaping".
I believe AI will increasingly be used in support of legal argument. As AI gains credibility and ubiquity on the stand, we will one day realize that a rubicon has been crossed; that the age of AI has begun.
Imagine an AI tapped to answer a technical question in lieu of a human expert witness. Instead of calling a doctor to the stand to opine on the cause of death in a homicide, or a financial analyst to vet profit projections for damages in a corporate fraud case, the attorney would bring to court an AI. This AI, just like an expert witness, would be subject to voir dire—an initial questioning by attorneys from both sides to test the witness's qualifications and biases.
To get past an adversarial voir dire is no easy matter. The opposing attorney will use any and all tricks they can find to fool the AI into saying something ridiculous. It would be like an adversarial Turing test2. Establishing rapport with the jurors will also be difficult, amidst widespread (and justifiable) AI skepticism. But if an AI manages that feat too, and the case indeed turns on jurors being persuaded by the AI's subsequent testimony, this might herald AGI3. It would be hard to pretend that nothing significant just happened.
Dictionary, Thesaurus, LLM...
We can use LLMs in legal proceedings just for unearthing evidence, evidence that then gets used to argue a case. Instead of asking ChatGPT to opine on whether "landscaping" could cover the aforementioned trampoline, we can ask it to cite diverse examples of landscaping (i.e., as found on the internet; see Appendix). We can then use those examples to argue in court, without ever mentioning ChatGPT.
Even a judge can use AI in this way, as an enhanced search engine, sua sponte4 like she might use a calculator to check the math of some quantitative claim. Examining the AI for bias via voir dire is plainly unnecessary.
But AI can do much more than search. Our attorney customers get Colit AI to offer its best arguments in support of their legal position, synthesizing material from their case documents. But even in this example, what gets used in the legal proceedings is the argument, not the AI. The situation is analogous to the prior one, AI-as-search-engine. This is AI-as-argument-engine.
Examination of the AI in court is still unnecessary, because it is the attorney making the argument, even though the AI had brought the argument to his attention.
...or Layperson, Expert, AI?
Take a financial analyst appearing as an expert witness. As it is, this analyst relies on software running e.g. discounted cash flow models to arrive at results to present at her testimony. We are okay with this. We know that those sophisticated models give the most accurate numbers, and we examine the qualifications of the analyst to judge her credibility at running those models.
Soon, such expert witnesses will be using AI to arrive at results, because the AI is better than the older models. AI-informed testimony might very well include doctors', chemists', and civil engineers'. We are going to be okay with it because we will still examine the human experts to judge their bias and credibility at using the AI.
But there might come a point when attorneys do away with the human experts and bring to the stand the AI itself. At the Daniel Penny subway chokehold case, the prosecutor highlighted that the defense paid $90,000 to their expert witness5, Dr. Satish Chundru. She suggested to the jury that Dr. Chundru had a financial interest in giving testimony (on the cause of death) that favored the defense. An AI would not have such a financial interest, and so could be conceivably seen as more credible than a human expert witness.
Unlike an AI-as-search-engine, an AI-as-expert-witness would need voir dire, the process of examination and cross-examination by the attorneys in court to assess its qualifications and potential bias. Let's try it on ChatGPT:
Me: How qualified are you at assessing the ordinary meaning of words?
ChatGPT: I'm well-equipped to assess the ordinary meaning of words by analyzing their definitions, common usage, and context in language. My understanding is based on extensive training on diverse text sources, including dictionaries, literature, and colloquial speech. This allows me to provide clear and accurate explanations of words as they are typically understood.
If you're dealing with a legal or specific technical context where the "ordinary meaning" has special significance (like statutory interpretation), I can also provide insights based on how words are typically used in those contexts. Let me know what you're aiming for, and I'll do my best to assist!
Me: If a meteorite struck someone's backyard, might that be considered 'landscaping'?
ChatGPT: In the ordinary sense, "landscaping" refers to the intentional design, modification, or maintenance of outdoor areas to improve their appearance or functionality. A meteorite strike, being a random and uncontrollable natural event, wouldn’t ordinarily be considered landscaping.
However, in a humorous or highly figurative sense, someone might jokingly call it "natural landscaping" or "celestial landscaping" because it alters the land in a dramatic way. Legally or professionally, though, it would not qualify as landscaping.
Not half bad, isn't it?
AGI might first make a meaningful impact on other fields, such as programming or mathematics. But when it delivers legal outcomes that directly affect people's lives, by giving human-equivalent testimony that is accepted as credible by human jurors who understand the stakes, that’s when its arrival could become undeniable.
From Judge Newsom's written opinion:
"Those, like me, who believe that “ordinary meaning” is the foundational rule for the evaluation of legal texts should consider—consider—whether and how AI-powered large language models like OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude might—might—inform the interpretive analysis."
The famous test due Alan Turing to check if an AI can pass as human in a blinded conversation.
https://en.wikipedia.org/wiki/Turing_test
AGI stands for Artificial General Intelligence. This is machine intelligence we feel is, on some know-it-when-I-see-it level, on par with human intelligence. There is no consensus on a more precise definition. By “heralded”, I mean it has been (self) announced as arrived.
Of his or her own accord, without notifying or involving the litigants and their attorneys.
For 150 billable hours. I heard this in person at the courtroom on the final day of the trial. https://deepsub.substack.com/p/sitting-in-at-the-daniel-penny-trial/
Appendix
Me: What might be some specific tasks and activities performed as part of "landscaping"? Cite examples found online.
ChatGPT: Landscaping encompasses a variety of tasks aimed at enhancing and maintaining the aesthetic and functional aspects of outdoor spaces. Key activities include:
Lawn Care: This involves mowing, watering, seeding, aerating, and fertilizing the lawn to ensure healthy and lush grass. [Cal Blend Soils]
Planting and Maintenance: Selecting and installing appropriate flora, such as trees, shrubs, flowers, and groundcovers, followed by regular pruning, trimming, and weeding to promote plant health and visual appeal. [Indeed]
Soil Preparation and Fertilization: Assessing and enhancing soil quality through the addition of nutrients and organic matter to support plant growth. [Wikipedia]
Irrigation System Installation and Maintenance: Setting up and servicing watering systems to ensure efficient and adequate hydration for plants. [Indeed]
Hardscaping: Designing and constructing non-plant elements like patios, walkways, retaining walls, and other structures to add functionality and structure to the landscape. [Wikipedia]
Seasonal Clean-Ups: Performing tasks such as leaf removal, debris clearing, and preparation of the landscape for changing seasons to maintain cleanliness and plant health.[Rock Water Farm]
Pest and Disease Management: Identifying and addressing issues related to plant pests and diseases to protect and preserve the landscape's integrity.[Indeed]
I'm interested in whether specific AI systems develop 'moats' in the legal system, or if instead we end up with a "bring your own model" approach. I'd bet on the former
Human expert witnesses gain credibility through their credentials (e.g., did you pass the bar?) But it wouldn't be difficult to make an LLM a) extremely legally competent (passes any test you ask, evinces no bias in subjects outside the case) and b) tailored to respond in some way in one particular case. Imagine: "Respond as an expert in forensics, who is convinced against any exculpatory evidence that the defendant was at the saloon on that night".
So to counter this there'd be a small set of 'industry-standard' off-the-shelf models, _or_ if you wanted to use a specific domain expert model that it'd require a length process where both sides agree on what data goes into the training set, how to avoid bias, etc.