DONT DO THIS. U.S. Lawyer got ChatGPT to write the submissions and it fabricated authority citations, notary fraud.

Warning to all who thing using AI to do their jobs for them. Same goes for students.

Finally, (one of) the most worrisome of ChatGPT / AI finally hit a jobs/business which (arguably) “very human and maybe never ever irreplaceable”: lawyer. After at least 24 universities (all universities between Top 1-Top 300 THES QS rating) reportedly found “cheated using AI” by students, now about judicial. ChatGPT making up fraud citations, notary fraud, and an incandescent the U.S. federal judge.

A lawyer named Steven Schwartz (read entire in here) got ChatGPT to write the submissions (plainly nonsense) and in doing so, it fabricated authority citations. When the (angry) Judge requested the cases, the lawyer went back and got ChatGPT to FABRICATE THE CASES. There’s lots of talk about how lawyers can make use of AI in their practice.

This is not a good example. It’s an impressively terrible example. Don’t do this. Warning to all who thing using AI to do their jobs for them! Same goes for students.

Professor Flunks All His Students After ChatGPT Falsely Claims It Wrote Their Papers. Texas A&M University (ranked 164th in QS World University Rankings 2023) –Commerce seniors who have already graduated were denied their diplomas because of an instructor who incorrectly used AI software to detect cheating.

The snafu began after Schwartz’s legal partner Paul LoDuca filed a lawsuit against the Colombian airline Avianca on behalf of Robert Mata, who was allegedly injured when a metal serving cart struck his knee on a flight to New York City.

When the airline’s lawyers asked the court to toss the suit, Schwartz filed a brief that supposedly cited more than a half dozen relevant cases.

Peter Loduca wrote last week in an affidavit he was not involved in the malfeasance and “had no reason to doubt the authenticity of the case law” fabricated in the document. 

Loduca added he had worked with Schwartz for 25 years and never recalled him looking to “mislead” a court.

AI can’t replace writing masterpiece of movies or series

Because fraud by ChatGPT, it appears Lawyer profession is safe for the time being. ChatGPT makes cases up - then fabricates the cases themselves when the lawyer is pushed to prove they exist. Stop being surprised that ChatGPT makes stuff up. It's not a knowledge engine. It's a very good autocomplete engine. It has no idea that it's getting stuff wrong, or indeed right.

ChatGPT makes up fake but plausible academic citations also. Known as ‘hallucitations’. Sometimes AI may be the worst mistake you make. ChatGPT will even give you fake hyperlinks to these fake cases that look legit too. E.g., ask it to provide 10 peer-reviewed sources on any topic, with hyperlinks and proper citations, and you'll get garbage that looks real.

If you ask ChatGPT to write a brief, it will spit out fake cases with facts and everything. Opposing counsel’s cases didn’t exist on Lexis, but when we asked ChatGPT if the cases exist, it said yes and gave the same facts. One lawyer in the U.S. says that two attorneys at her lawfirm had opposing counsel file ChatGPT briefs with fake cases this past week. AI is notorious for lying currently. The whole trick to AI is that’s it’s not actually intelligent; it’s just supposed to trick you, and convince you into thinking it is, and it works for most people; and is is getting even better at bulshitting every day.

This is spectacular. Every lawyer old enough to have learned to Shepherdize with books will laugh about this. This case will be used in ethics and Legal Research & Writing (LRW) courses for years.

lawyer got ChatGPT to write the submissions (plainly nonsense) and in doing so, it fabricated authority citations. When the (angry) Judge requested the cases, the lawyer went back and got ChatGPT to FABRICATE THE CASES. Document in here. Curious about what the AI came up with for the Durden vs. KLM case, and this paragraph leaped out on me. Is it citing itself?

scrap the U.S. Barristers Act & go in order of LSAT score if we all start drafting this way lol

AT LEAST, New Zealand Law Society recently sent out a warning noting that people had been contacting Society law libraries looking for cases that ChatGPT had made up. Maybe following other countries. Wondering whether in future we’re going to see courts require some sort of “AI Disclaimer” requirement or declaration/disclosure. I know that if I had wasted time trying to find cases that didn’t exist, I would be pissed & wanting sanctions. It’s both impressive and scary how good ChatGPT is at making up believable case cites.

There is so much about this that exposes ignorance about AI and how it works. I’m pretty ignorant, too, but I know enough to know that AI can’t (yet) tell if it’s right or wrong. It can only deduce patterns and (if it is generative) generate an answer that looks right. I asked ChatGPT a question of law and a question of fact. It got both wrong. The law question it got wrong because it had evidently not been trained on a dataset that included the right answer (or at least not enough iterations of it). I tried several different formulations. The factual answer was wrong because a key phrase in the question misled it into generating a response that had a statistically high chance of being right (because of how that phrase is and has for over 30 years most commonly been used), but wasn’t. AI is neither intelligent (in a human sense) nor omniscient. It’s capable of producing fantastic tools. But in the wrong hands it can also make you look like a tool. Add laziness and dishonesty and you get disastrous results.

It’s both impressive and scary how good ChatGPT is at making up believable case cites.

(Promoting to more engage in Substack) Seamless to listen to your favorite podcasts on Substack. You can buy a better headset to listen to a podcast here (GST DE352306207). Listeners on Apple Podcasts, Spotify, Overcast, or Pocket Casts simultaneously. podcasting can transform more of a conversation. Invite listeners to weigh in on episodes directly with you and with each other through discussion threads. At Substack, the process is to build with writers. Podcasts are an amazing feature of the Substack. I wish it had a feature to read the words we have written down without us having to do the speaking.

——

-prada- (Adi Mulia Pradana) is a Helper. Former adviser (President Indonesia) Jokowi for mapping 2-times election. I used to get paid to catch all these blunders—now I do it for free. Trying to work out what's going on, what happens next. Arch enemies of the tobacco industry, (still) survive after getting doxed.

(Very rare compliment and initiative pledge. Thank you. Yes, even a lot of people associated me PRAVDA, not part of MIUCCIA PRADA. I’m literally asshole on debate, since in college). Especially tonight heated between Putin and Prigozhin.

========

Thanks for reading Prada’s Newsletter. I was lured, inspired by someone writer, his post in LinkedIn months ago, “Currently after a routine daily writing newsletter in the last 10 years, my subscriber reaches 100,000. Maybe one of my subscribers is your boss.”  After I get followed / subscribed by (literally) prominent AI and prominent Chief Product and Technology of mammoth global media (both: Sir, thank you so much), I try crafting more / better writing.

To get the ones who really appreciate your writing, and now prominent people appreciate my writing, priceless feeling. Prada ungated/no paywall every notes-but thank you for anyone open initiative pledge to me.

(Promoting to more engage in Substack) Seamless to listen to your favorite podcasts on Substack. You can buy a better headset to listen to a podcast here (GST DE352306207). Listeners on Apple Podcasts, Spotify, Overcast, or Pocket Casts simultaneously. podcasting can transform more of a conversation. Invite listeners to weigh in on episodes directly with you and with each other through discussion threads. At Substack, the process is to build with writers. Podcasts are an amazing feature of the Substack. I wish it had a feature to read the words we have written down without us having to do the speaking. Thanks for reading Prada’s Newsletter.